smart_open API reference

115 public APIs from smart_open (piskvorky/smart_open) — 10 classes, 22 functions, 83 methods. Signatures extracted by static analysis of the actual source.

Repository: piskvorky/smart_open

Kind	Count
Classes	10
Functions	22
Methods	83

API list

funcrelease.update_changelog.get_json(url)

Perform a GET + json.loads.

classsmart_open.azure.AppendWriter

Append bytes to Azure Blob Storage.

methodsmart_open.azure.AppendWriter.closed() -> bool

Return True if the stream is closed.

methodsmart_open.azure.AppendWriter.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.azure.AppendWriter.seek(offset:int, whence:int=smart_open.constants.WHENCE_START) -> int

Unsupported.

methodsmart_open.azure.AppendWriter.tell() -> int

Return the current stream position.

methodsmart_open.azure.AppendWriter.terminate() -> None

AppendBlob cannot be aborted, so we do nothing here.

methodsmart_open.azure.AppendWriter.truncate(size:int | None=None) -> int

Unsupported.

methodsmart_open.azure.AppendWriter.writable() -> bool

Return True if the stream supports writing.

methodsmart_open.azure.AppendWriter.write(b:ReadableBuffer) -> int

Append `b` to the AppendBlob, buffering until ``min_part_size``.

classsmart_open.azure.Reader

Reads bytes from Azure Blob Storage.

methodsmart_open.azure.Reader.close() -> None

Flush and close this stream.

methodsmart_open.azure.Reader.closed() -> bool

Return True if the stream is closed.

methodsmart_open.azure.Reader.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.azure.Reader.read(size:int | None=-1) -> bytes

Read up to size bytes from the object and return them.

methodsmart_open.azure.Reader.read1(size:int | None=-1) -> bytes

This is the same as read().

methodsmart_open.azure.Reader.readable() -> bool

Return True if the stream can be read from.

methodsmart_open.azure.Reader.readinto(b:WriteableBuffer) -> int

Read up to len(b) bytes into b, and return the number of bytes read.

methodsmart_open.azure.Reader.readline(limit:int | None=-1) -> bytes

Read up to and including the next newline.

methodsmart_open.azure.Reader.seek(offset:int, whence:int=smart_open.constants.WHENCE_START) -> int

Seek to the specified position.

methodsmart_open.azure.Reader.seekable() -> bool

Return True; we support `seek` but not `truncate`.

methodsmart_open.azure.Reader.tell() -> int

Return the current position within the file.

methodsmart_open.azure.Reader.truncate(size:int | None=None) -> int

Unsupported.

classsmart_open.azure.Writer

Writes bytes to Azure Blob Storage.

methodsmart_open.azure.Writer.close() -> None

Commit the buffered block list and close the stream.

methodsmart_open.azure.Writer.closed() -> bool

Return True if the stream is closed.

methodsmart_open.azure.Writer.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.azure.Writer.seek(offset:int, whence:int=smart_open.constants.WHENCE_START) -> int

Unsupported.

methodsmart_open.azure.Writer.tell() -> int

Return the current stream position.

methodsmart_open.azure.Writer.terminate() -> None

Do not commit block list on abort.

methodsmart_open.azure.Writer.truncate(size:int | None=None) -> int

Unsupported.

methodsmart_open.azure.Writer.writable() -> bool

Return True if the stream supports writing.

methodsmart_open.azure.Writer.write(b:ReadableBuffer) -> int

Write the given bytes (binary string) to the Azure Blob Storage file.

funcsmart_open.azure.parse_uri(uri_as_string:str) -> _AzureUri

Parse an ``azure://`` URI into its container and blob components.

classsmart_open.concurrency.ThreadPoolExecutor

Subclass with a lazy consuming imap method.

funcsmart_open.concurrency.ThreadPoolExecutor.get() -> Any

Block until the next task is done and return the result.

funcsmart_open.doctools.extract_examples_from_readme(indent:str | None=None) -> str

Extract examples from this project's README.md file.

funcsmart_open.doctools.to_docstring(kwargs:list[Any], lpad:str='') -> str

Reconstruct a docstring from keyword argument info.

funcsmart_open.ftp.open_uri(uri:str, mode:str, transport_params:TransportParams) -> IO[Any]

Open an FTP/FTPS URI using the given mode and transport params.

funcsmart_open.ftp.parse_uri(uri_as_string:str) -> _FTPUri

Parse an ``ftp://`` or ``ftps://`` URI into connection components.

funcsmart_open.gcs.open_uri(uri:str, mode:str, transport_params:TransportParams) -> io.IOBase

Open a GCS URI using the given mode and transport params.

methodsmart_open.hdfs.CliRawInputBase.close() -> None

Flush and close this stream.

methodsmart_open.hdfs.CliRawInputBase.closed() -> bool

Return True if the stream is closed.

methodsmart_open.hdfs.CliRawInputBase.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.hdfs.CliRawInputBase.read(size:int | None=-1) -> bytes

Read up to size bytes from the object and return them.

methodsmart_open.hdfs.CliRawInputBase.read1(size:int | None=-1) -> bytes

This is the same as read().

methodsmart_open.hdfs.CliRawInputBase.readable() -> bool

Return True if the stream can be read from.

methodsmart_open.hdfs.CliRawInputBase.seekable() -> bool

Return False; HDFS streams do not support seeking.

methodsmart_open.hdfs.CliRawOutputBase.close() -> None

Flush and close this stream.

methodsmart_open.hdfs.CliRawOutputBase.closed() -> bool

Return True if the stream is closed.

methodsmart_open.hdfs.CliRawOutputBase.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.hdfs.CliRawOutputBase.flush() -> None

Flush the underlying ``hdfs dfs -put`` subprocess stdin.

methodsmart_open.hdfs.CliRawOutputBase.seekable() -> bool

Return False; HDFS streams do not support seeking.

methodsmart_open.hdfs.CliRawOutputBase.write(b:ReadableBuffer) -> int

Write the given buffer to the underlying raw stream.

methodsmart_open.hdfs.CliRawOutputBase.writeable() -> bool

Return True if this object is writeable.

funcsmart_open.hdfs.open(uri:str, mode:str) -> CliRawInputBase | CliRawOutputBase

Open an HDFS `uri` for reading or writing.

funcsmart_open.hdfs.parse_uri(uri_as_string:str) -> _HDFSUri

Parse an ``hdfs://`` or ``viewfs://`` URI into its path component.

classsmart_open.http.SeekableBufferedInputBase

Seekable streamed reader from a web site.

methodsmart_open.http.SeekableBufferedInputBase.seek(offset:int, whence:int=0) -> int

Seek to the specified position.

methodsmart_open.http.SeekableBufferedInputBase.tell() -> int

Return the current stream position.

methodsmart_open.http.SeekableBufferedInputBase.truncate(size:int | None=None) -> int

Unsupported.

funcsmart_open.http.parse_uri(uri_as_string:str) -> _HTTPUri

Parse an ``http://`` or ``https://`` URI into its path component.

funcsmart_open.local_file.extract_local_path(uri_as_string:str) -> str

Return the user-expanded local filesystem path from `uri_as_string`.

funcsmart_open.local_file.open_uri(uri_as_string:str, mode:str, transport_params:TransportParams) -> IO[Any]

Open a local file URI using the given mode.

classsmart_open.s3.MultipartWriter

Writes bytes to S3 using the multi part API.

methodsmart_open.s3.MultipartWriter.closed() -> bool

Return True if the stream is closed.

methodsmart_open.s3.MultipartWriter.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.s3.MultipartWriter.seek(offset:int, whence:int=constants.WHENCE_START) -> int

Unsupported.

methodsmart_open.s3.MultipartWriter.tell() -> int

Return the current stream position.

methodsmart_open.s3.MultipartWriter.terminate() -> None

Cancel the underlying multipart upload.

methodsmart_open.s3.MultipartWriter.to_boto3(resource:Any) -> Any

Create an **independent** `boto3.s3.Object` instance.

methodsmart_open.s3.MultipartWriter.truncate(size:int | None=None) -> int

Unsupported.

methodsmart_open.s3.MultipartWriter.writable() -> bool

Return True if the stream supports writing.

methodsmart_open.s3.MultipartWriter.write(b:Buffer) -> int

Write the given buffer to the S3 file.

classsmart_open.s3.Reader

Reads bytes from S3.

methodsmart_open.s3.Reader.close() -> None

Flush and close this stream.

methodsmart_open.s3.Reader.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.s3.Reader.read(size:int | None=-1) -> bytes

Read up to size bytes from the object and return them.

methodsmart_open.s3.Reader.read1(size:int | None=-1) -> bytes

This is the same as read().

methodsmart_open.s3.Reader.readable() -> bool

Return True if the stream can be read from.

methodsmart_open.s3.Reader.readinto(b:WriteableBuffer) -> int

Read up to len(b) bytes into b, and return the number of bytes read.

methodsmart_open.s3.Reader.readline(limit:int | None=-1) -> bytes

Read up to and including the next newline.

methodsmart_open.s3.Reader.seek(offset:int, whence:int=constants.WHENCE_START) -> int

Seek to the specified position.

methodsmart_open.s3.Reader.seekable() -> bool

If False, seek(), tell() and truncate() will raise IOError.

methodsmart_open.s3.Reader.tell() -> int

Return the current position within the file.

methodsmart_open.s3.Reader.terminate() -> None

Do nothing.

methodsmart_open.s3.Reader.to_boto3(resource:Any) -> Any

Create an **independent** `boto3.s3.Object` instance.

methodsmart_open.s3.Reader.truncate(size:int | None=None) -> int

Unsupported.

classsmart_open.s3.SinglepartWriter

Writes bytes to S3 using the single part API.

methodsmart_open.s3.SinglepartWriter.closed() -> bool

Return True if the stream is closed.

methodsmart_open.s3.SinglepartWriter.flush() -> None

No-op flush; data is buffered until `close`.

methodsmart_open.s3.SinglepartWriter.read(size:int | None=-1) -> bytes

Propagate.

methodsmart_open.s3.SinglepartWriter.read1(size:int | None=-1) -> bytes

Propagate.

methodsmart_open.s3.SinglepartWriter.readable() -> bool

Propagate.

methodsmart_open.s3.SinglepartWriter.seek(offset:int, whence:int=constants.WHENCE_START) -> int

Propagate.

methodsmart_open.s3.SinglepartWriter.seekable() -> bool

Propagate.

methodsmart_open.s3.SinglepartWriter.tell() -> int

Propagate.

methodsmart_open.s3.SinglepartWriter.terminate() -> None

Close buffer and skip upload.

methodsmart_open.s3.SinglepartWriter.truncate(size:int | None=None) -> int

Propagate.

methodsmart_open.s3.SinglepartWriter.writable() -> bool

Propagate.

methodsmart_open.s3.SinglepartWriter.write(b:Buffer) -> int

Write the given buffer into the in-memory buffer.

funcsmart_open.s3.open_uri(uri:str, mode:str, transport_params:TransportParams) -> io.BufferedIOBase

Open an S3 URI using the given mode and transport params.

funcsmart_open.smart_open_lib.parse_uri(uri_as_string:str) -> tuple[Any, ...]

Parse the given URI from a string.

classsmart_open.smart_open_lib.patch_pathlib

Replace `Path.open` with `smart_open.open`.

funcsmart_open.transport.get_transport(scheme:str) -> ModuleType

Get the submodule that handles transport for the specified scheme.

funcsmart_open.utils.check_kwargs(kallable:Callable[..., Any], kwargs:dict[str, Any]) -> dict[str, Any]

Check which keyword arguments the callable supports.

funcsmart_open.utils.clamp(value:int, minval:int=0, maxval:int | None=None) -> int

Clamp a numeric value to a specific range.

funcsmart_open.utils.make_range_string(start:int | None=None, stop:int | None=None) -> str

Create a byte range specifier in accordance with RFC-2616.

classsmart_open.webhdfs.BufferedOutputBase

Writes bytes to a WebHDFS file in multipart chunks.

methodsmart_open.webhdfs.BufferedOutputBase.closed() -> bool

Return True if the stream is closed.

methodsmart_open.webhdfs.BufferedOutputBase.detach() -> io.RawIOBase

Unsupported.

methodsmart_open.webhdfs.BufferedOutputBase.writable() -> bool

Return True if the stream supports writing.

funcsmart_open.webhdfs.convert_to_http_uri(parsed_uri:Any) -> str

Convert a parsed webhdfs URI to its HTTP REST URL (compat wrapper).

func

smart_open.webhdfs.open(http_uri:str, mode:str, min_part_size:int=MIN_PART_SIZE) -> BufferedInputBase | BufferedOutputBase

Open a WebHDFS URI for reading or writing.

funcsmart_open.webhdfs.parse_uri(uri_as_str:str) -> _WebHDFSUri

Return the WebHDFS URI as a dict with `scheme` and `uri` keys.

About this data

These signatures were extracted from the public source of piskvorky/smart_open using Python's ast module. Argument names, default values, type annotations and return types are taken verbatim from the code. Implementation bodies are never stored. See how it works for details.

Back to all 805 libraries