Skip to content

data_pool

Module to interact with data pools.

This module contains class to interact with a data pool in EMS Data integration.

Typical usage example 
data_pool = celonis.data_integration.get_data_pool(data_pool_id)
data_pool.name = "NEW_NAME"
data_pool.update()
data_pool.delete()

DataPool

Bases: DataPoolTransport

Data pool object to interact with data pool specific data integration endpoints.

client class-attribute 

client: Client = Field(Ellipsis, exclude=True)

id class-attribute 

id: str

from_transport classmethod 

from_transport(client, data_pool_transport)

Creates high-level data pool object from given DataPoolTransport.

Parameters:

  • client (Client) –

    Client to use to make API calls for given data pool.

  • data_pool_transport (DataPoolTransport) –

    DataPoolTransport object containing properties of data pool.

Returns:

  • DataPool –

    A DataPool object with properties from transport and given client.

update 

update()

Pushes local changes of data pool to EMS and updates properties with response from EMS.

sync 

sync()

Syncs data pool properties with EMS.

delete 

delete()

Deletes data pool.

create_data_model 

create_data_model(name, **kwargs)

Creates new data model with name in given data pool.

Parameters:

  • name (str) –

    Name of new data model.

  • **kwargs (typing.Any) –

    Additional parameters set for DataModelTransport object.

Returns:

  • DataModel –

    A DataModel object for newly created data model.

get_data_model 

get_data_model(id_)

Gets data model with given id.

Parameters:

  • id_ (str) –

    Id of data model.

Returns:

  • DataModel –

    A DataModel object for data model with given id.

get_data_models 

get_data_models()

Gets all data models of given data pool.

Returns:

create_data_push_job_from staticmethod 

create_data_push_job_from(
    client,
    data_pool_id,
    target_name,
    type_=None,
    column_config=None,
    keys=None,
    **kwargs
)

Creates new data push job in given data pool.

Parameters:

  • client (Client) –

    Client to use to make API calls for data export.

  • data_pool_id (str) –

    Id of data pool where data push job will be created.

  • target_name (str) –

    Table name to which job will push data.

  • type_ (typing.Optional[JobType]) –

    Type of data push job.

  • column_config (typing.Optional[typing.List[ColumnTransport]]) –

    Can be used to specify column types and string field length in number of characters.

  • keys (typing.Optional[typing.List[str]]) –

    Primary keys to use in case of upsert data push job.

  • **kwargs (typing.Any) –

    Additional parameters set for DataPushJob object.

Returns:

create_data_push_job 

create_data_push_job(
    target_name,
    type_=None,
    column_config=None,
    keys=None,
    **kwargs
)

Creates new data push job in given data pool.

Parameters:

  • target_name (str) –

    Table name to which job will push data.

  • type_ (typing.Optional[JobType]) –

    Type of data push job.

  • column_config (typing.Optional[typing.List[ColumnTransport]]) –

    Can be used to specify column types and string field length in number of characters.

  • keys (typing.Optional[typing.List[str]]) –

    Primary keys to use in case of upsert data push job.

  • **kwargs (typing.Any) –

    Additional parameters set for DataPushJob object.

Returns:

  • DataPushJob –

    A DataPushJob object for newly created data push job.

get_data_push_job 

get_data_push_job(id_)

Gets data push job with given id.

Parameters:

  • id_ (str) –

    Id of data push job.

Returns:

  • DataPushJob –

    A DataPushJob object for data push job with given id.

get_data_push_jobs 

get_data_push_jobs()

Gets all data push jobs of given data pool.

Returns:

create_table 

create_table(
    df,
    table_name,
    drop_if_exists=False,
    column_config=None,
    chunk_size=100000,
    force=False,
    data_source_id=None,
    **kwargs
)

Creates new table in given data pool.

Parameters:

  • df (pd.DataFrame) –

    DataFrame to push to new table.

  • table_name (str) –

    Name of new table.

  • drop_if_exists (bool) –

    If true, drops existing table if it exists. If false, raises PyCelonisTableAlreadyExistsError if table already exists.

  • column_config (typing.Optional[typing.List[ColumnTransport]]) –

    Can be used to specify column types and string field length in number of characters.

  • chunk_size (int) –

    Number of rows to push in one chunk.

  • force (bool) –

    If true, replacing table without column config is possible. Otherwise, error is raised if table would be replaced without column config.

  • data_source_id (typing.Optional[str]) –

    Data source id of table

  • **kwargs (typing.Any) –

    Additional parameters set for DataPushJob object.

Returns:

Raises:

  • PyCelonisTableAlreadyExistsError –

    Raised if drop_if_exists=False and table already exists.

  • PyCelonisDataPushExecutionFailedError –

    Raised when table creation fails.

  • PyCelonisValueError –

    Raised when table already exists and no column config is given.

get_table 

get_table(name, data_source_id=None)

Gets table located in data pool with given name and data source id.

Parameters:

  • name (str) –

    Name of table.

  • data_source_id (typing.Optional[str]) –

    Data source id of table

Returns:

Raises:

get_tables 

get_tables()

Gets all data pool tables of given data pool.

Returns:

get_data_connection 

get_data_connection(id_)

Gets data connection with given id.

Parameters:

  • id_ (str) –

    Id of data connection.

Returns:

  • DataConnection –

    A DataConnection object for data connection with given id.

get_data_connections 

get_data_connections()

Gets all data connections of given data pool.

Returns:

  • CelonisCollection[DataConnection] –

    A list containing all data connections.

create_job 

create_job(name, data_source_id=None, **kwargs)

Creates new job with name in given data pool.

Parameters:

  • name (str) –

    Name of new job.

  • data_source_id (typing.Optional[str]) –

    Id of data source to use for job scope.

Returns:

  • Job –

    A Job object for newly created job.

get_job 

get_job(id_)

Gets job with given id.

Parameters:

  • id_ (str) –

    Id of job.

Returns:

  • Job –

    A Job object for job with given id.

get_jobs 

get_jobs()

Gets all jobs of given data pool.

Returns: