data_export

Module to interact with data exports.

This module contains class to interact with a data export request in EMS data integration.

Typical usage example:`

```python
# Export data directly as data frame:
df = data_model.export_data_frame(
    query
)

# Export data to CSV files
data_export = data_model.create_data_export(
    query,
    ExportType.CSV,
)
data_export.wait_for_execution()

for (
    i,
    chunk,
) in enumerate(
    data_export.get_chunks()
):
    with open(
        f"FILE_PATH_{i}.csv",
        "wb",
    ) as f:
        f.write(
            chunk.read()
        )
```

DataExport

Bases: DataExportStatusResponse

Data export object to interact with data export specific data integration endpoints.

export_status class-attribute instance-attribute

export_status = Field(None, alias='exportStatus')

created class-attribute instance-attribute

created = Field(None, alias='created')

message class-attribute instance-attribute

message = Field(None, alias='message')

export_type class-attribute instance-attribute

export_type = Field(None, alias='exportType')

export_chunks class-attribute instance-attribute

export_chunks = Field(None, alias='exportChunks')

client class-attribute instance-attribute

client = Field(..., exclude=True)

id instance-attribute

id

Id of data export.

data_model_id instance-attribute

data_model_id

Id of data model where data export is located.

from_transport classmethod

from_transport(
    client, data_model_id, data_export_transport
)

Creates high-level data export object from given DataExportStatusResponse.

Parameters:

• client (Client) –

  Client to use to make API calls for given data pool.

• data_model_id (str) –

  Id of data model where data export is located.

• data_export_transport (DataExportStatusResponse) –

  DataExportStatusResponse object containing properties of data export.

Returns:

• DataExport –

  A DataExport object with properties from transport and given client.

get_chunks

get_chunks()

Yields exported chunks.

Yields:

• BytesIO –

  Byte chunks of exported data.

Raises:

• PyCelonisDataExportRunningError –

  Raised in case export is still in progress.

• PyCelonisDataExportFailedError –

  Raised in case export failed.

wait_for_execution

wait_for_execution()

Waits until data export execution is done.

Examples:

Manually run data export and wait for it to finish:

data_export = data_model.create_data_export(
    query=query,
    export_type=ExportType.PARQUET,
)
data_export.wait_for_execution()
chunks = data_export.get_chunks()

for chunk in chunks:
    with open(
        f"<file_name>.parquet",
        "wb",
    ) as f:
        f.write(
            chunk.read()
        )

DataExportV2

Bases: ProxyDataExportStatusResponseV2

Data export object to interact with version 2 of data export specific data integration endpoints.

created class-attribute instance-attribute

created = Field(None, alias='created')

export_status class-attribute instance-attribute

export_status = Field(None, alias='exportStatus')

export_type class-attribute instance-attribute

export_type = Field(None, alias='exportType')

exported_chunks class-attribute instance-attribute

exported_chunks = Field(None, alias='exportedChunks')

messages class-attribute instance-attribute

messages = Field(None, alias='messages')

client class-attribute instance-attribute

client = Field(..., exclude=True)

id instance-attribute

id

Id of data export.

data_model_id instance-attribute

data_model_id

Id of data model where data export is located.

from_transport classmethod

from_transport(
    client, data_model_id, data_export_transport
)

Creates high-level data export object from given DataExportStatusResponse.

Parameters:

• client (Client) –

  Client to use to make API calls for given data pool.

• data_model_id (str) –

  Id of data model where data export is located.

• data_export_transport (ProxyDataExportStatusResponseV2) –

  ProxyDataExportStatusResponseV2 object containing properties of data export.

Returns:

• DataExportV2 –

  A DataExport object with properties from transport and given client.

get_chunks

get_chunks()

Yields exported chunks.

Yields:

• BytesIO –

  Byte chunks of exported data.

wait_for_execution

wait_for_execution()

Waits until data export execution is done.

Examples:

Manually run data export and wait for it to finish to get all chunks:

data_export = data_model.create_data_export(
    query=query,
    export_type=ExportType.PARQUET,
)
data_export.wait_for_execution()
chunks = data_export.get_chunks()

for chunk in chunks:
    with open(
        f"<file_name>.parquet",
        "wb",
    ) as f:
        f.write(
            chunk.read()
        )