datasets API reference

207 public APIs from datasets (huggingface/datasets) — 79 classes, 36 functions, 92 methods. Signatures extracted by static analysis of the actual source.

Repository: huggingface/datasets

Kind	Count
Classes	79
Functions	36
Methods	92

API list

classsrc.datasets.arrow_dataset.Column

An iterable for a specific column of a [`Dataset`].

classsrc.datasets.arrow_dataset.Dataset

A Dataset backed by an Arrow table.

method

src.datasets.arrow_dataset.Dataset.add_column(name:str, column:Union[list, np.ndarray], new_fingerprint:Optional[str]=None, feature:Optional[FeatureType]=None)

Add column to Dataset.

methodsrc.datasets.arrow_dataset.Dataset.add_item(item:dict, new_fingerprint:Optional[str]=None)

Add item to Dataset.

method

src.datasets.arrow_dataset.Dataset.cast_column(column:str, feature:FeatureType, new_fingerprint:Optional[str]=None) -> 'Dataset'

Cast column to feature for decoding.

methodsrc.datasets.arrow_dataset.Dataset.column_names() -> list[str]

Names of the columns in the dataset.

methodsrc.datasets.arrow_dataset.Dataset.data() -> Table

The Apache Arrow table backing the dataset.

methodsrc.datasets.arrow_dataset.Dataset.flatten(new_fingerprint:Optional[str]=None, max_depth=16) -> 'Dataset'

Flatten the table.

methodsrc.datasets.arrow_dataset.Dataset.num_columns() -> int

Number of columns in the dataset.

methodsrc.datasets.arrow_dataset.Dataset.skip(n:int) -> 'Dataset'

Create a new [`Dataset`] that skips the first `n` elements.

methodsrc.datasets.arrow_dataset.Dataset.take(n:int) -> 'Dataset'

Create a new [`Dataset`] with only the first `n` elements.

method

src.datasets.arrow_dataset.Dataset.to_dict(batch_size:Optional[int]=None, batched:bool=False) -> Union[dict, Iterator[dict]]

Returns the dataset as a Python dict.

methodsrc.datasets.arrow_dataset.Dataset.to_list() -> list

Returns the dataset as a Python list.

method

src.datasets.arrow_dataset.Dataset.to_pandas(batch_size:Optional[int]=None, batched:bool=False) -> Union[pd.DataFrame, Iterator[pd.DataFrame]]

Returns the dataset as a `pandas.DataFrame`.

methodsrc.datasets.arrow_dataset.Dataset.unique(column:str) -> list

Return a list of the unique elements in a column.

classsrc.datasets.arrow_dataset.NonExistentDatasetError

Used when we expect the existence of a dataset

classsrc.datasets.arrow_reader.ArrowReader

Build a Dataset object out of Instruction instance(s).

methodsrc.datasets.arrow_reader.ArrowReader.read_table(filename, in_memory=False) -> Table

Read table from file.

classsrc.datasets.arrow_reader.BaseReader

Build a Dataset object out of Instruction instance(s).

methodsrc.datasets.arrow_reader.BaseReader.read(name, instructions, split_infos, in_memory=False)

Returns Dataset instance(s).

classsrc.datasets.arrow_reader.ParquetReader

Build a Dataset object out of Instruction instance(s).

classsrc.datasets.arrow_reader.ReadInstruction

Reading instruction for a dataset.

classsrc.datasets.arrow_writer.ArrowWriter

Shuffles and writes Examples to Arrow files.

methodsrc.datasets.arrow_writer.ArrowWriter.write_table(pa_table:pa.Table, writer_batch_size:Optional[int]=None)

Write a Table to file.

classsrc.datasets.builder.BuilderConfig

Base class for `DatasetBuilder` data configuration.

methodsrc.datasets.builder.BuilderConfig.create_config_id(config_kwargs:dict, custom_features:Optional[Features]=None) -> str

The config id is used to build the cache directory.

classsrc.datasets.builder.DatasetBuilder

Abstract base class for all datasets.

method

src.datasets.builder.DatasetBuilder.as_dataset(split:Optional[Union[str, Split, list[str], list[Split]]]=None, in_memory=False) -> Union[Dataset, DatasetDict]

Return a Dataset for the specified split.

classsrc.datasets.data_files.DataFilesList

List of data files (absolute local paths or URLs).

classsrc.datasets.download.download_config.DownloadConfig

Configuration for our cached path manager.

classsrc.datasets.exceptions.DataFilesNotFoundError

No (supported) data files found.

classsrc.datasets.exceptions.DatasetNotFoundError

Dataset not found.

classsrc.datasets.exceptions.DatasetsError

Base class for exceptions in this library.

classsrc.datasets.exceptions.DefunctDatasetError

The dataset has been defunct.

classsrc.datasets.exceptions.ExpectedMoreSplitsError

Some recorded splits are missing.

classsrc.datasets.exceptions.FileNotFoundDatasetsError

FileNotFoundError raised by this library.

classsrc.datasets.exceptions.SplitsVerificationError

Error raised during splits verifications.

classsrc.datasets.exceptions.UnexpectedDownloadedFileError

Some downloaded files were not expected.

classsrc.datasets.exceptions.UnexpectedSplitsError

The expected splits of the downloaded file is missing.

classsrc.datasets.features.audio.Audio

Audio [`Feature`] to extract audio data from an audio file.

methodsrc.datasets.features.audio.Audio.cast_storage(storage:Union[pa.StringArray, pa.StructArray]) -> pa.StructArray

Cast an Arrow array to the Audio arrow storage type.

method

src.datasets.features.audio.Audio.decode_example(value:dict, token_per_repo_id:Optional[dict[str, Union[str, bool, None]]]=None) -> 'AudioDecoder'

Decode example audio file into audio data.

method

src.datasets.features.audio.Audio.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed audio files into the Arrow array.

methodsrc.datasets.features.audio.Audio.encode_example(value:Union[str, bytes, bytearray, dict, 'AudioDecoder']) -> dict

Encode example into a format for Arrow.

classsrc.datasets.features.features.Array2D

Create a two-dimensional array.

classsrc.datasets.features.features.Array3D

Create a three-dimensional array.

classsrc.datasets.features.features.Array4D

Create a four-dimensional array.

classsrc.datasets.features.features.Array5D

Create a five-dimensional array.

classsrc.datasets.features.features.ClassLabel

Feature type for integer class labels.

methodsrc.datasets.features.features.ClassLabel.int2str(values:Union[int, Iterable]) -> Union[str, Iterable]

Conversion `integer` => class name `string`.

methodsrc.datasets.features.features.ClassLabel.str2int(values:Union[str, Iterable]) -> Union[int, Iterable]

Conversion class name `string` => `integer`.

methodsrc.datasets.features.features.Features.arrow_schema()

Features schema.

methodsrc.datasets.features.features.Features.copy() -> 'Features'

Make a deep copy of [`Features`].

methodsrc.datasets.features.features.Features.encode_column(column, column_name:str)

Encode column into a format for Arrow.

methodsrc.datasets.features.features.Features.flatten(max_depth=16) -> 'Features'

Flatten the features.

methodsrc.datasets.features.features.Features.from_arrow_schema(pa_schema:pa.Schema) -> 'Features'

Construct [`Features`] from Arrow Schema.

methodsrc.datasets.features.features.Features.from_dict(dic) -> 'Features'

Construct [`Features`] from dict.

methodsrc.datasets.features.features.Features.type()

Features field types.

classsrc.datasets.features.features.Json

Feature type for JSON objects.

methodsrc.datasets.features.features.Json.cast_storage(storage:Union[pa.Array]) -> pa.JsonArray

Cast an Arrow array to the `Json` arrow storage type.

classsrc.datasets.features.features.Value

Scalar feature value of a particular data type.

func

src.datasets.features.features.decode_nested_example(schema, obj, token_per_repo_id:Optional[dict[str, Union[str, bool, None]]]=None)

Decode a nested example.

funcsrc.datasets.features.features.encode_nested_example(schema, obj, level=0)

Encode a nested example.

funcsrc.datasets.features.features.require_decoding(feature:FeatureType, ignore_decode_attribute:bool=False) -> bool

Check if a (possibly nested) feature requires decoding.

funcsrc.datasets.features.features.to_pyarrow_listarray(data:Any, pa_type:_ArrayXDExtensionType) -> pa.Array

Convert to PyArrow ListArray.

classsrc.datasets.features.image.Image

Image [`Feature`] to read image data from an image file.

methodsrc.datasets.features.image.Image.decode_example(value:dict, token_per_repo_id=None) -> 'PIL.Image.Image'

Decode example image file into image data.

method

src.datasets.features.image.Image.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed image files into the Arrow array.

method

src.datasets.features.image.Image.encode_example(value:Union[str, bytes, bytearray, dict, np.ndarray, 'PIL.Image.Image']) -> dict

Encode example into a format for Arrow.

classsrc.datasets.features.mesh.Mesh

Mesh [`Feature`] to read 3D mesh data from a file.

methodsrc.datasets.features.mesh.Mesh.cast_storage(storage:Union[pa.StringArray, pa.StructArray]) -> pa.StructArray

Cast an Arrow array to the Mesh arrow storage type.

method

src.datasets.features.mesh.Mesh.decode_example(value:dict, token_per_repo_id=None) -> Union['trimesh.Trimesh', 'trimesh.Scene']

Decode example mesh file.

method

src.datasets.features.mesh.Mesh.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed mesh files into the Arrow array.

method

src.datasets.features.mesh.Mesh.encode_example(value:Union[str, bytes, bytearray, dict, 'trimesh.Trimesh', 'trimesh.Scene']) -> dict

Encode example into a format for Arrow.

method

src.datasets.features.pdf.Pdf.cast_storage(storage:Union[pa.StringArray, pa.StructArray, pa.ListArray]) -> pa.StructArray

Cast an Arrow array to the Pdf arrow storage type.

methodsrc.datasets.features.pdf.Pdf.decode_example(value:dict, token_per_repo_id=None) -> 'pdfplumber.pdf.PDF'

Decode example pdf file into pdf data.

method

src.datasets.features.pdf.Pdf.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed PDF files into the Arrow array.

methodsrc.datasets.features.pdf.Pdf.encode_example(value:Union[str, bytes, bytearray, dict, 'pdfplumber.pdf.PDF']) -> dict

Encode example into a format for Arrow.

funcsrc.datasets.features.pdf.encode_pdfplumber_pdf(pdf:'pdfplumber.pdf.PDF') -> dict

Encode a pdfplumber.pdf.PDF object into a dictionary.

funcsrc.datasets.features.pdf.pdf_to_bytes(pdf:'pdfplumber.pdf.PDF') -> bytes

Convert a pdfplumber.pdf.PDF object to bytes.

classsrc.datasets.features.video.Video

Video [`Feature`] to read video data from a video file.

method

src.datasets.features.video.Video.decode_example(value:Union[str, Example], token_per_repo_id:Optional[dict[str, Union[bool, str]]]=None) -> 'VideoDecoder'

Decode example video file into video data.

method

src.datasets.features.video.Video.embed_storage(storage:pa.StructArray, token_per_repo_id=None, local_files:bool=True, remote_files:bool=True) -> pa.StructArray

Embed image files into the Arrow array.

method

src.datasets.features.video.Video.encode_example(value:Union[str, bytes, bytearray, Example, np.ndarray, 'VideoDecoder']) -> Example

Encode example into a format for Arrow.

funcsrc.datasets.filesystems.is_remote_filesystem(fs:fsspec.AbstractFileSystem) -> bool

Checks if `fs` is a remote filesystem.

funcsrc.datasets.filesystems.rename(fs:fsspec.AbstractFileSystem, src:str, dst:str)

Renames the file `src` in `fs` to `dst`.

classsrc.datasets.fingerprint.Hasher

Hasher that accepts python objects as inputs.

classsrc.datasets.formatting.formatting.LazyDict

A dictionary backed by Arrow data.

classsrc.datasets.info.DatasetInfo

Information about a dataset.

classsrc.datasets.info.NonMatchingCachedSizesError

The prepared split doesn't have expected sizes.

classsrc.datasets.io.spark.SparkDatasetReader

A dataset reader that reads from a Spark DataFrame.

classsrc.datasets.iterable_dataset.IterableDataset

A Dataset backed by an iterable.

methodsrc.datasets.iterable_dataset.IterableDataset.add_column(name:str, column:Union[list, np.array]) -> 'IterableDataset'

Add column to Dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.cast(features:Features) -> 'IterableDataset'

Cast the dataset to a new set of features.

methodsrc.datasets.iterable_dataset.IterableDataset.cast_column(column:str, feature:FeatureType) -> 'IterableDataset'

Cast column to feature for decoding.

methodsrc.datasets.iterable_dataset.IterableDataset.column_names() -> Optional[list[str]]

Names of the columns in the dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.load_state_dict(state_dict:dict) -> None

Load the state_dict of the dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.num_columns() -> Optional[int]

Number of columns in the dataset.

methodsrc.datasets.iterable_dataset.IterableDataset.reshard() -> 'IterableDataset'

Reshard the dataset if possible, i.e.

methodsrc.datasets.iterable_dataset.IterableDataset.state_dict() -> dict

Get the current state_dict of the dataset.

method

src.datasets.iterable_dataset.IterableDataset.to_csv(path_or_buf:Union[PathLike, BinaryIO], batch_size:Optional[int]=None, storage_options:Optional[dict]=None, **to_csv_kwargs) -> int

Exports the dataset to csv.

method

src.datasets.iterable_dataset.IterableDataset.to_dict(batch_size:Optional[int]=None, batched:bool=False) -> Union[dict, Iterator[dict]]

Returns the dataset as a Python dict.

methodsrc.datasets.iterable_dataset.IterableDataset.to_list() -> list

Returns the dataset as a Python list.

methodsrc.datasets.iterable_dataset.IterableDataset.with_format(type:Optional[str]=None) -> 'IterableDataset'

Return a dataset with the specified format.

funcsrc.datasets.load.increase_load_count(name:str)

Update the download count of a dataset.

classsrc.datasets.packaged_modules.arrow.arrow.ArrowConfig

BuilderConfig for Arrow.

classsrc.datasets.packaged_modules.audiofolder.audiofolder.AudioFolderConfig

Builder Config for AudioFolder.

classsrc.datasets.packaged_modules.conll.conll.ConllConfig

BuilderConfig for CoNLL-style files.

classsrc.datasets.packaged_modules.csv.csv.CsvConfig

BuilderConfig for CSV.

classsrc.datasets.packaged_modules.folder_based_builder.folder_based_builder.FolderBasedBuilderConfig

BuilderConfig for AutoFolder.

classsrc.datasets.packaged_modules.hdf5.hdf5.HDF5Config

BuilderConfig for HDF5.

classsrc.datasets.packaged_modules.iceberg.iceberg.IcebergConfig

BuilderConfig for Apache Iceberg format.

classsrc.datasets.packaged_modules.imagefolder.imagefolder.ImageFolderConfig

BuilderConfig for ImageFolder.

classsrc.datasets.packaged_modules.json.json.JsonConfig

BuilderConfig for JSON.

classsrc.datasets.packaged_modules.lance.lance.LanceConfig

BuilderConfig for Lance format.

classsrc.datasets.packaged_modules.meshfolder.meshfolder.MeshFolderConfig

BuilderConfig for MeshFolder.

classsrc.datasets.packaged_modules.niftifolder.niftifolder.NiftiFolderConfig

BuilderConfig for NiftiFolder.

classsrc.datasets.packaged_modules.pandas.pandas.PandasConfig

BuilderConfig for Pandas.

classsrc.datasets.packaged_modules.parquet.parquet.ParquetConfig

BuilderConfig for Parquet.

classsrc.datasets.packaged_modules.pdffolder.pdffolder.PdfFolderConfig

BuilderConfig for ImageFolder.

classsrc.datasets.packaged_modules.spark.spark.SparkConfig

BuilderConfig for Spark.

classsrc.datasets.packaged_modules.sql.sql.SqlConfig

BuilderConfig for SQL.

classsrc.datasets.packaged_modules.text.text.TextConfig

BuilderConfig for text files.

classsrc.datasets.packaged_modules.videofolder.videofolder.VideoFolderConfig

BuilderConfig for ImageFolder.

funcsrc.datasets.packaged_modules.webdataset._tenbin.str64(s)

Convert a string to an int64.

funcsrc.datasets.packaged_modules.webdataset._tenbin.unstr64(i)

Convert an int64 to a string.

classsrc.datasets.packaged_modules.xml.xml.XmlConfig

BuilderConfig for xml files.

classsrc.datasets.search.BaseIndex

Base class for indexing

methodsrc.datasets.search.BaseIndex.load(file:Union[str, PurePath]) -> 'BaseIndex'

Deserialize the index from disk

methodsrc.datasets.search.BaseIndex.save(file:Union[str, PurePath])

Serialize the index on disk

methodsrc.datasets.search.BaseIndex.search(query, k:int=10, **kwargs) -> SearchResults

To implement.

methodsrc.datasets.search.BaseIndex.search_batch(queries, k:int=10, **kwargs) -> BatchedSearchResults

Find the nearest examples indices to the query.

classsrc.datasets.search.ElasticSearchIndex

Sparse index using Elasticsearch.

methodsrc.datasets.search.ElasticSearchIndex.add_documents(documents:Union[list[str], 'Dataset'], column:Optional[str]=None)

Add documents to the index.

methodsrc.datasets.search.ElasticSearchIndex.search(query:str, k=10, **kwargs) -> SearchResults

Find the nearest examples indices to the query.

classsrc.datasets.search.FaissIndex

Dense index using Faiss.

method

src.datasets.search.FaissIndex.load(file:Union[str, PurePath], device:Optional[Union[int, list[int]]]=None, storage_options:Optional[dict]=None) -> 'FaissIndex'

Deserialize the FaissIndex from disk

methodsrc.datasets.search.FaissIndex.save(file:Union[str, PurePath], storage_options:Optional[dict]=None)

Serialize the FaissIndex on disk

methodsrc.datasets.search.FaissIndex.search(query:np.array, k=10, **kwargs) -> SearchResults

Find the nearest examples indices to the query.

methodsrc.datasets.search.FaissIndex.search_batch(queries:np.array, k=10, **kwargs) -> BatchedSearchResults

Find the nearest examples indices to the queries.

classsrc.datasets.search.IndexableMixin

Add indexing features to `datasets.Dataset`

methodsrc.datasets.search.IndexableMixin.drop_index(index_name:str)

Drop the index with the specified column.

methodsrc.datasets.search.IndexableMixin.get_index(index_name:str) -> BaseIndex

List the `index_name`/identifiers of all the attached indexes.

method

src.datasets.search.IndexableMixin.save_faiss_index(index_name:str, file:Union[str, PurePath], storage_options:Optional[dict]=None)

Save a FaissIndex on disk.

classsrc.datasets.splits.NamedSplit

Descriptor corresponding to a named split (train, test, ...).

classsrc.datasets.splits.Split

`Enum` for dataset splits.

classsrc.datasets.splits.SplitBase

Abstract base class for Split compositionality.

classsrc.datasets.splits.SplitDict

Split info object.

methodsrc.datasets.splits.SplitDict.add(split_info:SplitInfo)

Add the split info.

classsrc.datasets.splits.SplitGenerator

Defines the split information for the generator.

classsrc.datasets.splits.SplitReadInstruction

Object containing the reading instruction for the dataset.

classsrc.datasets.splits.SubSplitInfo

Wrapper around a sub split info.

classsrc.datasets.table.InMemoryTable

The table is said in-memory when it is loaded into the user's RAM.

methodsrc.datasets.table.InMemoryTable.add_column(*args, **kwargs)

Add column to Table at position.

methodsrc.datasets.table.InMemoryTable.filter(*args, **kwargs)

Select records from a Table.

methodsrc.datasets.table.InMemoryTable.flatten(*args, **kwargs)

Flatten this Table.

methodsrc.datasets.table.InMemoryTable.from_arrays(*args, **kwargs)

Construct a Table from Arrow arrays.

methodsrc.datasets.table.InMemoryTable.select(*args, **kwargs)

Select columns of the table.

classsrc.datasets.table.Table

Wraps a pyarrow Table by using composition.

methodsrc.datasets.table.Table.add_column(*args, **kwargs)

Add column to Table at position.

methodsrc.datasets.table.Table.append_column(*args, **kwargs)

Append column at end of columns.

methodsrc.datasets.table.Table.cast(*args, **kwargs)

Cast table values to another schema.

methodsrc.datasets.table.Table.column_names()

Names of the table's columns.

methodsrc.datasets.table.Table.columns()

List of all columns in numerical order.

methodsrc.datasets.table.Table.filter(*args, **kwargs)

Select records from a Table.

methodsrc.datasets.table.Table.flatten(*args, **kwargs)

Flatten this Table.

methodsrc.datasets.table.Table.num_columns()

Number of columns in this table.

methodsrc.datasets.table.Table.num_rows()

Number of rows in this table.

methodsrc.datasets.table.Table.schema()

Schema of the table and its columns.

methodsrc.datasets.table.Table.select(*args, **kwargs)

Select columns of the table.

methodsrc.datasets.table.Table.set_column(*args, **kwargs)

Replace column in Table at position.

methodsrc.datasets.table.Table.slice(*args, **kwargs)

Compute zero-copy slice of this Table.

methodsrc.datasets.table.Table.to_reader(max_chunksize:Optional[int]=None)

Convert the Table to a RecordBatchReader.

methodsrc.datasets.table.Table.validate(*args, **kwargs)

Perform validation checks.

funcsrc.datasets.table.cast_table_to_schema(table:pa.Table, schema:pa.Schema)

Cast a table to the arrow schema.

funcsrc.datasets.table.concat_tables(tables:list[Table], axis:int=0) -> Table

Concatenate tables.

funcsrc.datasets.table.list_table_cache_files(table:Table) -> list[str]

Get the cache files that are loaded by the table.

funcsrc.datasets.table.table_cast(table:pa.Table, schema:pa.Schema)

Improved version of `pa.Table.cast`.

funcsrc.datasets.table.table_flatten(table:pa.Table)

Improved version of `pa.Table.flatten`.

funcsrc.datasets.table.table_iter(table:Table, batch_size:int, drop_last_batch=False) -> Iterator[pa.Table]

Iterate over sub-tables of size `batch_size`.

classsrc.datasets.utils._dataset_viewer.DatasetViewerError

Dataset viewer error.

funcsrc.datasets.utils._dill.dump(obj, file)

Pickle an object to a file.

funcsrc.datasets.utils._dill.dumps(obj)

Pickle an object to a string.

classsrc.datasets.utils._filelock.FileLock

A `filelock.FileLock` initializer that handles long paths.

funcsrc.datasets.utils.experimental.experimental(fn:Callable) -> Callable

Decorator to flag a feature as experimental.

classsrc.datasets.utils.file_utils.FilesIterable

An iterable of paths from a list of directories or files

funcsrc.datasets.utils.file_utils.get_authentication_headers_for_url(url:str, token:Optional[Union[str, bool]]=None) -> dict

Handle the HF authentication

funcsrc.datasets.utils.file_utils.relative_to_absolute_path(path:T) -> T

Convert relative path to absolute path.

funcsrc.datasets.utils.file_utils.xgetsize(path, download_config:Optional[DownloadConfig]=None) -> int

Extend `os.path.getsize` function to support remote files.

funcsrc.datasets.utils.file_utils.xisdir(path, download_config:Optional[DownloadConfig]=None) -> bool

Extend `os.path.isdir` function to support remote files.

funcsrc.datasets.utils.file_utils.xisfile(path, download_config:Optional[DownloadConfig]=None) -> bool

Extend `os.path.isfile` function to support remote files.

funcsrc.datasets.utils.file_utils.xlistdir(path:str, download_config:Optional[DownloadConfig]=None) -> list[str]

Extend `os.listdir` function to support remote files.

classsrc.datasets.utils.info_utils.VerificationMode

`Enum` that specifies which verification checks to run.

funcsrc.datasets.utils.info_utils.get_size_checksum_dict(path:str, record_checksum:bool=False) -> dict

Compute the file size and the sha256 checksum of a file

funcsrc.datasets.utils.logging.disable_propagation() -> None

Disable propagation of the library log outputs.

funcsrc.datasets.utils.logging.enable_propagation() -> None

Enable propagation of the library log outputs.

funcsrc.datasets.utils.logging.get_logger(name:Optional[str]=None) -> logging.Logger

Return a logger with the specified name.

funcsrc.datasets.utils.logging.set_verbosity(verbosity:int) -> None

Set the level for the Hugging Face Datasets library's root logger.

classsrc.datasets.utils.metadata.MetadataConfigs

Should be in format {config_name: {**config_params}}.

classsrc.datasets.utils.py_utils.NonMutableDict

Dict where keys can only be added but not modified.

classsrc.datasets.utils.py_utils.classproperty

Descriptor to be used as decorator for @classmethods.

funcsrc.datasets.utils.py_utils.string_to_dict(string:str, pattern:str) -> Optional[dict[str, str]]

Un-format a string using a python f-string pattern.

funcsrc.datasets.utils.py_utils.temp_seed(seed:int, set_pytorch=False, set_tensorflow=False)

Temporarily set the random seed.

classsrc.datasets.utils.version.Version

Dataset version `MAJOR.MINOR.PATCH`.

funcutils.release.get_version()

Reads the current version in the __init__.

funcutils.release.global_version_update(version)

Update the version in all needed files.

funcutils.release.post_release_work()

Do all the necesarry post-release steps.

About this data

These signatures were extracted from the public source of huggingface/datasets using Python's ast module. Argument names, default values, type annotations and return types are taken verbatim from the code. Implementation bodies are never stored. See how it works for details.

Back to all 805 libraries