Dataset module

DatasetActions

Bases: BaseAction

Dataset actions

append_to_dataset(project_id, files, batch_id=None, dataset_id=None, ignore_exist_errors=False, disable_progress_bar=False)

Add files to a dataset

Info

You must provide a dataset_id or a batch_id to use the dataset linked to the batch

Parameters:

Name	Type	Description	Default
project_id	str	ID of the project	required
dataset_id	Optional[str]	ID of the dataset_id	None
batch_id	Optional[str]	ID of the batch	None
files	List[Union[Dict, str, Tuple[str, IOBase]]]	list of the file to create, str or Dict -> FilePayload(path : str, file : str)	required
disable_progress_bar	Optional[bool]	Disable the progress bar display	False

Returns:

Type	Description
None	None

browse_dataset(dataset_id, folder=None, disable_progress_bar=False, iterate=False)

Browse dataset

Parameters:

Name	Type	Description	Default
dataset_id	str	ID of the dataset	required
folder	Optional[str]	Base folder to start browsing	None
disable_progress_bar	Optional[bool]	Disable the progress bar display,	False
iterate	Optional[bool]	Return a generator	False

Returns:

Type	Description
Union[Generator[Iterable[str], None, None], Iterable[str]]	List (or Generator) of project representations

create_dataset(project_id, name, integration_id=None, base_folder=None, disable_progress_bar=False)

Create an integration

Parameters:

Name	Type	Description	Default
project_id	ProjectId	Project ID of the dataset	required
name	str	Name of the dataset	required
integration_id	Optional[IntegrationId]	Integration ID of the dataset	None
base_folder	Optional[str]	If integration, specify a subfolder in the integration bucket	None
disable_progress_bar	Optional[bool]	Disable the progress bar display	False

IntegrationsActions

Bases: BaseAction

Integrations actions

create_integration(name, type, visibility, bucket_access_point, role_id=None, external_id=None, disable_progress_bar=False)

Create an integration

Parameters:

Name	Type	Description	Default
name	str	Name of the batch	required
type	IntegrationType	Type of the integration, Possible choices: GCP, S3.	required
visibility	IntegrationVisibility	Visibility of the integration Possible choices: organization, private.	required
bucket_access_point	str	Access point for S3 or bucket for GCP	required
role_id	Optional[str]	Role ID or ARN for S3 integration	None
external_id	Optional[str]	Client external ID for S3 integration	None
disable_progress_bar	Optional[bool]	Disable the progress bar display	False

integrations(visibility_in=None, search=None, disable_progress_bar=False, iterate=False)

Get the list of your integrations and those of your organization

Parameters:

Name	Type	Description	Default
visibility_in	Optional[List[IntegrationVisibility]]	Only in those visibilities. Possible choices: organization, private.	None
search	Optional[str]	Quicksearch,	None
disable_progress_bar	Optional[bool]	Disable the progress bar display,	False
iterate	Optional[bool]	Return a generator	False

Returns:

Type	Description
Union[Generator[Iterable[Dict], None, None], Iterable[Dict]]	List (or Generator) of integration representations