Metadata-Version: 2.4
Name: PowerPlatform-Dataverse-Client
Version: 0.1.0b10
Summary: Python SDK for Microsoft Dataverse
Author: Microsoft Corporation
License-Expression: MIT
Project-URL: Homepage, https://github.com/microsoft/PowerPlatform-DataverseClient-Python
Project-URL: Repository, https://github.com/microsoft/PowerPlatform-DataverseClient-Python.git
Project-URL: Issues, https://github.com/microsoft/PowerPlatform-DataverseClient-Python/issues
Project-URL: Documentation, https://github.com/microsoft/PowerPlatform-DataverseClient-Python#readme
Keywords: dataverse,powerapps,powerplatform,crm,dynamics,odata
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: azure-identity>=1.17.0
Requires-Dist: azure-core>=1.30.2
Requires-Dist: requests>=2.32.0
Requires-Dist: pandas>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Dynamic: license-file

# PowerPlatform Dataverse Client for Python

[![PyPI version](https://img.shields.io/pypi/v/PowerPlatform-Dataverse-Client.svg)](https://pypi.org/project/PowerPlatform-Dataverse-Client/)
[![Python](https://img.shields.io/pypi/pyversions/PowerPlatform-Dataverse-Client.svg)](https://pypi.org/project/PowerPlatform-Dataverse-Client/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A Python client library for Microsoft Dataverse that provides a unified interface for CRUD operations, SQL queries, table metadata management, and file uploads through the Dataverse Web API.

**[Source code](https://github.com/microsoft/PowerPlatform-DataverseClient-Python)** | **[Package (PyPI)](https://pypi.org/project/PowerPlatform-Dataverse-Client/)** | **[API reference documentation](https://learn.microsoft.com/python/api/dataverse-sdk-docs-python/dataverse-overview?view=dataverse-sdk-python-latest)** | **[Product documentation](https://learn.microsoft.com/power-apps/developer/data-platform/sdk-python/)** | **[Samples](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/tree/main/examples)**

> [!IMPORTANT]
> This library is currently in **preview**. Preview versions are provided for early access to new features and may contain breaking changes.

## Table of contents

- [Key features](#key-features)
- [Getting started](#getting-started)
  - [Prerequisites](#prerequisites)
  - [Install the package](#install-the-package)  
  - [Authenticate the client](#authenticate-the-client)
- [Key concepts](#key-concepts)
- [Examples](#examples)
  - [Quick start](#quick-start)
  - [Basic CRUD operations](#basic-crud-operations)
  - [Bulk operations](#bulk-operations)
  - [Upsert operations](#upsert-operations)
  - [DataFrame operations](#dataframe-operations)
  - [Query data](#query-data) *(QueryBuilder, SQL, raw OData)*
  - [Table management](#table-management)
  - [Relationship management](#relationship-management)
  - [File operations](#file-operations)
  - [Batch operations](#batch-operations)
- [Next steps](#next-steps)
- [Troubleshooting](#troubleshooting)
- [Contributing](#contributing)

## Key features

- **🔄 CRUD Operations**: Create, read, update, and delete records with support for bulk operations and automatic retry
- **⚡ True Bulk Operations**: Automatically uses Dataverse's native `CreateMultiple`, `UpdateMultiple`, `UpsertMultiple`, and `BulkDelete` Web API operations for maximum performance and transactional integrity
- **🔍 Fluent QueryBuilder**: Type-safe query construction with method chaining, composable filter expressions, and automatic OData generation
- **📊 SQL Queries**: Execute read-only SQL queries via the Dataverse Web API `?sql=` parameter
- **🏗️ Table Management**: Create, inspect, and delete custom tables and columns programmatically
- **🔗 Relationship Management**: Create one-to-many and many-to-many relationships between tables with full metadata control
- **🐼 DataFrame Support**: Pandas wrappers for all CRUD operations, returning DataFrames and Series
- **📎 File Operations**: Upload files to Dataverse file columns with automatic chunking for large files
- **📦 Batch Operations**: Send multiple CRUD, table metadata, and SQL query operations in a single HTTP request with optional transactional changesets
- **🔐 Azure Identity**: Built-in authentication using Azure Identity credential providers with comprehensive support
- **🛡️ Error Handling**: Structured exception hierarchy with detailed error context and retry guidance
- **📋 HTTP Diagnostics Logging**: Opt-in file-based logging of all HTTP requests and responses with automatic redaction of sensitive headers (e.g. `Authorization`)

## Getting started

### Prerequisites

- **Python 3.10+** (3.10, 3.11, 3.12, 3.13 supported)  
- **Microsoft Dataverse environment** with appropriate permissions
- **OAuth authentication configured** for your application

### Install the package

Install the PowerPlatform Dataverse Client using [pip](https://pypi.org/project/pip/):

```bash
# Install the latest stable release
pip install PowerPlatform-Dataverse-Client
```

(Optional) Install Claude Skill globally with the Client:

```bash
pip install PowerPlatform-Dataverse-Client && dataverse-install-claude-skill
```

This installs two Claude Skills that enable Claude Code to:
- **dataverse-sdk-use**: Apply SDK best practices for using the SDK in your applications
- **dataverse-sdk-dev**: Provide guidance for developing/contributing to the SDK itself

The skills work with both the Claude Code CLI and VSCode extension. Once installed, Claude will automatically use the appropriate skill when working with Dataverse operations. For more information on Claude Skill see https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview. See skill definitions here: [.claude/skills/dataverse-sdk-use/SKILL.md](.claude/skills/dataverse-sdk-use/SKILL.md) and [.claude/skills/dataverse-sdk-dev/SKILL.md](.claude/skills/dataverse-sdk-dev/SKILL.md).

For development from source (Claude Skill auto loaded):

```bash
git clone https://github.com/microsoft/PowerPlatform-DataverseClient-Python.git
cd PowerPlatform-DataverseClient-Python
pip install -e .
```

### Authenticate the client

The client requires any Azure Identity `TokenCredential` implementation for OAuth authentication with Dataverse:

```python
from azure.identity import (
    InteractiveBrowserCredential, 
    ClientSecretCredential,
    CertificateCredential,
    AzureCliCredential
)
from PowerPlatform.Dataverse.client import DataverseClient

# Development options
credential = InteractiveBrowserCredential()  # Browser authentication
# credential = AzureCliCredential()          # If logged in via 'az login'

# Production options  
# credential = ClientSecretCredential(tenant_id, client_id, client_secret)
# credential = CertificateCredential(tenant_id, client_id, cert_path)

client = DataverseClient("https://yourorg.crm.dynamics.com", credential)
```

> **Complete authentication setup**: See **[Use OAuth with Dataverse](https://learn.microsoft.com/power-apps/developer/data-platform/authenticate-oauth)** for app registration, all credential types, and security configuration.

## Key concepts

The SDK provides a simple, pythonic interface for Dataverse operations:

| Concept | Description |
|---------|-------------|
| **DataverseClient** | Main entry point; provides `records`, `query`, `tables`, `files`, and `batch` namespaces |
| **Context Manager** | Use `with DataverseClient(...) as client:` for automatic cleanup and HTTP connection pooling |
| **Namespaces** | Operations are organized into `client.records` (CRUD & OData queries), `client.query` (QueryBuilder & SQL), `client.tables` (metadata), `client.files` (file uploads), and `client.batch` (batch requests) |
| **Records** | Dataverse records represented as Python dictionaries with column schema names |
| **Schema names** | Use table schema names (`"account"`, `"new_MyTestTable"`) and column schema names (`"name"`, `"new_MyTestColumn"`). See: [Table definitions in Microsoft Dataverse](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/entity-metadata) |
| **Bulk Operations** | Efficient bulk processing for multiple records with automatic optimization |
| **Paging** | Automatic handling of large result sets with iterators |
| **Structured Errors** | Detailed exception hierarchy with retry guidance and diagnostic information |
| **Customization prefix values** | Custom tables and columns require a customization prefix value to be included for all operations (e.g., `"new_MyTestTable"`, not `"MyTestTable"`). See: [Table definitions in Microsoft Dataverse](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/entity-metadata) |

## Examples

### Quick start

```python
from azure.identity import InteractiveBrowserCredential
from PowerPlatform.Dataverse.client import DataverseClient

# Connect to Dataverse
credential = InteractiveBrowserCredential()

with DataverseClient("https://yourorg.crm.dynamics.com", credential) as client:
    # Create a contact
    contact_id = client.records.create("contact", {"firstname": "John", "lastname": "Doe"})

    # Read the contact back
    contact = client.records.get("contact", contact_id, select=["firstname", "lastname"])
    print(f"Created: {contact['firstname']} {contact['lastname']}")

    # Clean up
    client.records.delete("contact", contact_id)
# Session closed, caches cleared automatically
```

### Basic CRUD operations

```python
# Create a record
account_id = client.records.create("account", {"name": "Contoso Ltd"})

# Read a record
account = client.records.get("account", account_id)
print(account["name"])

# Update a record
client.records.update("account", account_id, {"telephone1": "555-0199"})

# Delete a record
client.records.delete("account", account_id)
```

### Bulk operations

```python
# Bulk create
payloads = [
    {"name": "Company A"},
    {"name": "Company B"},
    {"name": "Company C"}
]
ids = client.records.create("account", payloads)

# Bulk update (broadcast same change to all)
client.records.update("account", ids, {"industry": "Technology"})

# Bulk delete
client.records.delete("account", ids, use_bulk_delete=True)
```

### Upsert operations

Use `client.records.upsert()` to create or update records identified by alternate keys. When the
key matches an existing record it is updated; otherwise the record is created. A single item uses
a PATCH request; multiple items use the `UpsertMultiple` bulk action.

> **Prerequisite**: The table must have an **alternate key** configured in Dataverse for the
> columns used in `alternate_key`. Alternate keys are defined in the table's metadata (Power Apps
> maker portal → Table → Keys, or via the Dataverse API). Without a configured alternate key,
> upsert requests will be rejected by Dataverse with a 400 error.

```python
from PowerPlatform.Dataverse.models.upsert import UpsertItem

# Upsert a single record
client.records.upsert("account", [
    UpsertItem(
        alternate_key={"accountnumber": "ACC-001"},
        record={"name": "Contoso Ltd", "telephone1": "555-0100"},
    )
])

# Upsert multiple records (uses UpsertMultiple bulk action)
client.records.upsert("account", [
    UpsertItem(
        alternate_key={"accountnumber": "ACC-001"},
        record={"name": "Contoso Ltd"},
    ),
    UpsertItem(
        alternate_key={"accountnumber": "ACC-002"},
        record={"name": "Fabrikam Inc"},
    ),
])

# Composite alternate key (multiple columns identify the record)
client.records.upsert("account", [
    UpsertItem(
        alternate_key={"accountnumber": "ACC-001", "address1_postalcode": "98052"},
        record={"name": "Contoso Ltd"},
    )
])

# Plain dict syntax (no import needed)
client.records.upsert("account", [
    {
        "alternate_key": {"accountnumber": "ACC-001"},
        "record": {"name": "Contoso Ltd"},
    }
])
```

### DataFrame operations

The SDK provides pandas wrappers for all CRUD operations via the `client.dataframe` namespace, using DataFrames and Series for input and output.

```python
import pandas as pd

# Query records as a single DataFrame
df = client.dataframe.get("account", filter="statecode eq 0", select=["name", "telephone1"])
print(f"Found {len(df)} accounts")

# Limit results with top for large tables
df = client.dataframe.get("account", select=["name"], top=100)

# Fetch a single record as a one-row DataFrame
df = client.dataframe.get("account", record_id=account_id, select=["name"])

# Create records from a DataFrame (returns a Series of GUIDs)
new_accounts = pd.DataFrame([
    {"name": "Contoso", "telephone1": "555-0100"},
    {"name": "Fabrikam", "telephone1": "555-0200"},
])
new_accounts["accountid"] = client.dataframe.create("account", new_accounts)

# Update records from a DataFrame (id_column identifies the GUID column)
new_accounts["telephone1"] = ["555-0199", "555-0299"]
client.dataframe.update("account", new_accounts, id_column="accountid")

# Clear a field by setting clear_nulls=True (by default, NaN/None fields are skipped)
df = pd.DataFrame([{"accountid": new_accounts["accountid"].iloc[0], "websiteurl": None}])
client.dataframe.update("account", df, id_column="accountid", clear_nulls=True)

# Delete records by passing a Series of GUIDs
client.dataframe.delete("account", new_accounts["accountid"])

# SQL query directly to DataFrame (supports JOINs, aggregates, GROUP BY)
df = client.dataframe.sql(
    "SELECT a.name, COUNT(c.contactid) as contacts "
    "FROM account a "
    "JOIN contact c ON a.accountid = c.parentcustomerid "
    "GROUP BY a.name"
)
```

### Query data

The **QueryBuilder** is the recommended way to query records. It provides a fluent, type-safe interface that generates correct OData queries automatically — no need to remember OData filter syntax.

```python
# Fluent query builder (recommended)
for record in (client.query.builder("account")
               .select("name", "revenue")
               .filter_eq("statecode", 0)
               .filter_gt("revenue", 1000000)
               .order_by("revenue", descending=True)
               .top(100)
               .page_size(50)
               .execute()):
    print(f"{record['name']}: {record['revenue']}")
```

The QueryBuilder handles value formatting, column name casing, and OData syntax automatically. All filter methods are discoverable via IDE autocomplete:

```python
# Get results as a pandas DataFrame (consolidates all pages)
df = (client.query.builder("account")
      .select("name", "telephone1")
      .filter_eq("statecode", 0)
      .top(100)
      .to_dataframe())
print(f"Got {len(df)} accounts")
```

```python
# Comparison filters
query = (client.query.builder("contact")
         .filter_eq("statecode", 0)          # statecode eq 0
         .filter_gt("revenue", 1000000)      # revenue gt 1000000
         .filter_contains("name", "Corp")    # contains(name, 'Corp')
         .filter_in("statecode", [0, 1])     # Microsoft.Dynamics.CRM.In(...)
         .filter_between("revenue", 100000, 500000)  # (revenue ge 100000 and revenue le 500000)
         .filter_null("telephone1")          # telephone1 eq null
         )
```

For complex logic (OR, NOT, grouping), use the composable expression tree with `where()`:

```python
from PowerPlatform.Dataverse.models.filters import eq, gt, filter_in, between

# OR conditions: (statecode = 0 OR statecode = 1) AND revenue > 100k
for record in (client.query.builder("account")
               .select("name", "revenue")
               .where((eq("statecode", 0) | eq("statecode", 1))
                      & gt("revenue", 100000))
               .execute()):
    print(record["name"])

# NOT, between, and in operators
for record in (client.query.builder("account")
               .where(~eq("statecode", 2))                  # NOT inactive
               .where(between("revenue", 100000, 500000))    # revenue in range
               .execute()):
    print(record["name"])
```

**Formatted values and annotations** -- request localized labels, currency symbols, and display names:

```python
# Get formatted values (choice labels, currency, lookup names)
for record in (client.query.builder("account")
               .select("name", "statecode", "revenue")
               .include_formatted_values()
               .execute()):
    status = record["statecode@OData.Community.Display.V1.FormattedValue"]
    print(f"{record['name']}: {status}")
```

**Nested expand with options** -- expand navigation properties with `$select`, `$filter`, `$orderby`, and `$top`:

```python
from PowerPlatform.Dataverse.models.query_builder import ExpandOption

# Expand related tasks with filtering and sorting
for record in (client.query.builder("account")
               .select("name")
               .expand(ExpandOption("Account_Tasks")
                       .select("subject", "createdon")
                       .filter("contains(subject,'Task')")
                       .order_by("createdon", descending=True)
                       .top(5))
               .execute()):
    print(record["name"], record.get("Account_Tasks"))
```

**Record count** -- include `$count=true` in the request:

```python
# Request count alongside results
results = (client.query.builder("account")
           .filter_eq("statecode", 0)
           .count()
           .execute())
```

**SQL queries** provide an alternative read-only query syntax with support for
JOINs, aggregates, GROUP BY, DISTINCT, and OFFSET FETCH pagination:

```python
# Basic query
results = client.query.sql(
    "SELECT TOP 10 accountid, name FROM account WHERE statecode = 0"
)

# JOINs and aggregates work
results = client.query.sql(
    "SELECT a.name, COUNT(c.contactid) as cnt "
    "FROM account a "
    "JOIN contact c ON a.accountid = c.parentcustomerid "
    "GROUP BY a.name"
)

# SQL results directly as a DataFrame
df = client.dataframe.sql(
    "SELECT name, revenue FROM account ORDER BY revenue DESC"
)

# SQL helpers: discover columns and JOINs from metadata
cols = client.query.sql_select("account")  # "accountid, name, revenue, ..."
join = client.query.sql_join("contact", "account", from_alias="c", to_alias="a")
# Returns: "JOIN account a ON c.parentcustomerid = a.accountid"

# Build queries using helpers -- no OData knowledge needed
sql = f"SELECT TOP 10 c.fullname, a.name FROM contact c {join}"
df = client.dataframe.sql(sql)

# Discover all possible JOINs from a table (including polymorphic)
joins = client.query.sql_joins("opportunity")
for j in joins:
    print(f"{j['column']:30s} -> {j['target']}.{j['target_pk']}")
```

**Raw OData queries** are available via `records.get()` for cases where you need direct control over the OData filter string. The SDK provides helpers to eliminate the most error-prone parts:

```python
# Discover columns for $select (returns list ready for select= parameter)
cols = client.query.odata_select("account")
for page in client.records.get("account", select=cols, top=10):
    ...

# Discover $expand navigation properties (auto-resolves PascalCase names)
nav = client.query.odata_expand("contact", "account")
# Returns: "parentcustomerid_account"
for page in client.records.get("contact", select=["fullname"], expand=[nav], top=5):
    for r in page:
        acct = r.get(nav) or {}
        print(f"{r['fullname']} -> {acct.get('name')}")

# Build @odata.bind for lookup fields (no manual name construction)
bind = client.query.odata_bind("contact", "account", account_id)
# Returns: {"parentcustomerid_account@odata.bind": "/accounts(guid)"}
client.records.create("contact", {"firstname": "Jane", **bind})

# Raw OData query with manual parameters
for page in client.records.get(
    "account",
    select=["name"],
    filter="statecode eq 0",  # Raw OData: column names must be lowercase
    expand=["primarycontactid"],  # Navigation properties are case-sensitive
    top=100,
):
    for record in page:
        print(record["name"])
```

### Table management

```python
# Create a custom table, including the customization prefix value in the schema names for the table and columns.
table_info = client.tables.create("new_Product", {
    "new_Code": "string",
    "new_Description": "memo",
    "new_Price": "decimal",
    "new_Active": "bool"
})

# Create with custom primary column name and solution assignment
table_info = client.tables.create(
    "new_Product",
    columns={
        "new_Code": "string",
        "new_Price": "decimal"
    },
    solution="MyPublisher",  # Optional: add to specific solution
    primary_column="new_ProductName",  # Optional: custom primary column (default is "{customization prefix value}_Name")
)

# Get table information
info = client.tables.get("new_Product")
print(f"Logical name: {info['table_logical_name']}")
print(f"Entity set: {info['entity_set_name']}")

# List all tables
tables = client.tables.list()
for table in tables:
    print(table)

# Add columns to existing table (columns must include customization prefix value)
client.tables.add_columns("new_Product", {"new_Category": "string"})

# Remove columns
client.tables.remove_columns("new_Product", ["new_Category"])

# List all columns (attributes) for a table to discover schema
columns = client.tables.list_columns("account")
for col in columns:
    print(f"{col['LogicalName']} ({col.get('AttributeType')})")

# List only specific properties
columns = client.tables.list_columns(
    "account",
    select=["LogicalName", "SchemaName", "AttributeType"],
    filter="AttributeType eq 'String'",
)

# Clean up
client.tables.delete("new_Product")
```

> **Important**: All custom column names must include the customization prefix value (e.g., `"new_"`).
> This ensures explicit, predictable naming and aligns with Dataverse metadata requirements.

### Relationship management

Create relationships between tables using the relationship API. For a complete working example, see [examples/advanced/relationships.py](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/relationships.py).

```python
from PowerPlatform.Dataverse.models.relationship import (
    LookupAttributeMetadata,
    OneToManyRelationshipMetadata,
    ManyToManyRelationshipMetadata,
)
from PowerPlatform.Dataverse.models.labels import Label, LocalizedLabel

# Create a one-to-many relationship: Department (1) -> Employee (N)
# This adds a "Department" lookup field to the Employee table
lookup = LookupAttributeMetadata(
    schema_name="new_DepartmentId",
    display_name=Label(localized_labels=[LocalizedLabel(label="Department", language_code=1033)]),
)

relationship = OneToManyRelationshipMetadata(
    schema_name="new_Department_Employee",
    referenced_entity="new_department",   # Parent table (the "one" side)
    referencing_entity="new_employee",    # Child table (the "many" side)
    referenced_attribute="new_departmentid",
)

result = client.tables.create_one_to_many_relationship(lookup, relationship)
print(f"Created lookup field: {result['lookup_schema_name']}")

# Create a many-to-many relationship: Employee (N) <-> Project (N)
# Employees work on multiple projects; projects have multiple team members
m2m_relationship = ManyToManyRelationshipMetadata(
    schema_name="new_employee_project",
    entity1_logical_name="new_employee",
    entity2_logical_name="new_project",
)

result = client.tables.create_many_to_many_relationship(m2m_relationship)
print(f"Created M:N relationship: {result['relationship_schema_name']}")

# Query relationship metadata
rel = client.tables.get_relationship("new_Department_Employee")
if rel:
    print(f"Found: {rel['SchemaName']}")

# List all relationships
rels = client.tables.list_relationships()
for rel in rels:
    print(f"{rel['SchemaName']} ({rel.get('@odata.type')})")

# List relationships for a specific table (one-to-many + many-to-one + many-to-many)
account_rels = client.tables.list_table_relationships("account")
for rel in account_rels:
    print(f"{rel['SchemaName']} -> {rel.get('@odata.type')}")

# Delete a relationship
client.tables.delete_relationship(result['relationship_id'])
```

For simpler scenarios, use the convenience method:

```python
# Quick way to create a lookup field with sensible defaults
result = client.tables.create_lookup_field(
    referencing_table="contact",       # Child table gets the lookup field
    lookup_field_name="new_AccountId",
    referenced_table="account",        # Parent table being referenced
    display_name="Account",
)
```

### File operations

```python
# Upload a file to a record
client.files.upload(
    "account",
    account_id,
    "new_Document",  # If the file column doesn't exist, it will be created automatically
    "/path/to/document.pdf",
)
```

### Batch operations

Use `client.batch` to send multiple operations in one HTTP request. The batch namespace mirrors `client.records`, `client.tables`, and `client.query`.

```python
# Build a batch request and add operations
batch = client.batch.new()
batch.records.create("account", {"name": "Contoso"})
batch.records.create("account", [{"name": "Fabrikam"}, {"name": "Woodgrove"}])
batch.records.update("account", account_id, {"telephone1": "555-0100"})
batch.records.delete("account", old_id)
batch.records.get("account", account_id, select=["name"])

result = batch.execute()
for item in result.responses:
    if item.is_success:
        print(f"[OK] {item.status_code} entity_id={item.entity_id}")
    else:
        print(f"[ERR] {item.status_code}: {item.error_message}")
```

**Transactional changeset** — all operations in a changeset succeed or roll back together:

```python
batch = client.batch.new()
with batch.changeset() as cs:
    lead_ref = cs.records.create("lead", {"firstname": "Ada"})
    contact_ref = cs.records.create("contact", {"firstname": "Ada"})
    cs.records.create("account", {
        "name": "Babbage & Co.",
        "originatingleadid@odata.bind": lead_ref,
        "primarycontactid@odata.bind": contact_ref,
    })
result = batch.execute()
print(f"Created {len(result.entity_ids)} records atomically")
```

**Table metadata and SQL queries in a batch:**

```python
batch = client.batch.new()
batch.tables.create("new_Product", {"new_Price": "decimal", "new_InStock": "bool"})
batch.tables.add_columns("new_Product", {"new_Rating": "int"})
batch.tables.get("new_Product")
batch.query.sql("SELECT TOP 5 name FROM account")

result = batch.execute()
```

**Continue on error** — attempt all operations even when one fails:

```python
result = batch.execute(continue_on_error=True)
print(f"Succeeded: {len(result.succeeded)}, Failed: {len(result.failed)}")
for item in result.failed:
    print(f"[ERR] {item.status_code}: {item.error_message}")
```

**DataFrame integration** -- feed pandas DataFrames directly into a batch:

```python
import pandas as pd

batch = client.batch.new()

# Create records from a DataFrame
df = pd.DataFrame([{"name": "Contoso"}, {"name": "Fabrikam"}])
batch.dataframe.create("account", df)

# Update records from a DataFrame
updates = pd.DataFrame([
    {"accountid": id1, "telephone1": "555-0100"},
    {"accountid": id2, "telephone1": "555-0200"},
])
batch.dataframe.update("account", updates, id_column="accountid")

# Delete records from a Series
batch.dataframe.delete("account", pd.Series([id1, id2]))

result = batch.execute()
```

For a complete example see [examples/advanced/batch.py](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/batch.py).

## Next steps

### More sample code

Explore our comprehensive examples in the [`examples/`](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/tree/main/examples) directory:

**🌱 Getting Started:**
- **[Installation & Setup](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/basic/installation_example.py)** - Validate installation and basic usage patterns
- **[Functional Testing](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/basic/functional_testing.py)** - Test core functionality in your environment

**🚀 Advanced Usage:**
- **[Complete Walkthrough](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/walkthrough.py)** - Full feature demonstration with production patterns
- **[Relationship Management](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/relationships.py)** - Create and manage table relationships
- **[File Upload](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/file_upload.py)** - Upload files to Dataverse file columns
- **[Batch Operations](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/advanced/batch.py)** - Send multiple operations in a single request with changesets

📖 See the [examples README](https://github.com/microsoft/PowerPlatform-DataverseClient-Python/blob/main/examples/README.md) for detailed guidance and learning progression.

### Additional documentation

For comprehensive information on Microsoft Dataverse and related technologies:

| Resource | Description |
|----------|-------------|
| **[Dataverse Developer Guide](https://learn.microsoft.com/power-apps/developer/data-platform/)** | Complete developer documentation for Microsoft Dataverse |
| **[Dataverse Web API Reference](https://learn.microsoft.com/power-apps/developer/data-platform/webapi/)** | Detailed Web API reference and examples |  
| **[Azure Identity for Python](https://learn.microsoft.com/python/api/overview/azure/identity-readme)** | Authentication library documentation and credential types |
| **[Power Platform Developer Center](https://learn.microsoft.com/power-platform/developer/)** | Broader Power Platform development resources |
| **[Dataverse SDK for .NET](https://learn.microsoft.com/power-apps/developer/data-platform/org-service/overview)** | Official .NET SDK for Microsoft Dataverse |

## Troubleshooting

### General

The client raises structured exceptions for different error scenarios:

```python
from PowerPlatform.Dataverse.client import DataverseClient
from PowerPlatform.Dataverse.core.errors import HttpError, ValidationError

try:
    client.records.get("account", "invalid-id")
except HttpError as e:
    print(f"HTTP {e.status_code}: {e.message}")
    print(f"Error code: {e.code}")
    print(f"Subcode: {e.subcode}")
    if e.is_transient:
        print("This error may be retryable")
except ValidationError as e:
    print(f"Validation error: {e.message}")
```

### Authentication issues

**Common fixes:**
- Verify environment URL format: `https://yourorg.crm.dynamics.com` (no trailing slash)
- Ensure Azure Identity credentials have proper Dataverse permissions
- Check app registration permissions are granted and admin-consented

### Performance considerations

For optimal performance in production environments:

| Best Practice | Description |
|---------------|-------------|
| **Bulk Operations** | Pass lists to `records.create()`, `records.update()` for automatic bulk processing, for `records.delete()`, set `use_bulk_delete` when passing lists to use bulk operation |
| **Select Fields** | Specify `select` parameter to limit returned columns and reduce payload size |
| **Page Size Control** | Use `top` and `page_size` parameters to control memory usage |
| **Connection Reuse** | Reuse `DataverseClient` instances across operations |
| **Production Credentials** | Use `ClientSecretCredential` or `CertificateCredential` for unattended operations |
| **Error Handling** | Implement retry logic for transient errors (`e.is_transient`) |

### HTTP diagnostics logging

Enable file-based HTTP logging to capture all requests and responses for debugging. Sensitive headers (e.g. `Authorization`) are automatically redacted.

```python
from PowerPlatform.Dataverse.client import DataverseClient
from PowerPlatform.Dataverse.core.config import DataverseConfig
from PowerPlatform.Dataverse.core.log_config import LogConfig

log_cfg = LogConfig(
    log_folder="./my_logs",      # Directory for log files (created if missing)
    log_file_prefix="crm_debug", # Filename prefix; timestamp appended automatically
    max_body_bytes=4096,         # Bytes of body to capture per entry — 0 (default) disables body capture
)
config = DataverseConfig(log_config=log_cfg)
client = DataverseClient("https://yourorg.crm.dynamics.com", credential, config=config)
```

Each log file is timestamped and rotated automatically (default 10 MB per file, 5 backups). Sample output:

```
[2026-04-11T15:27:31-0700] DEBUG >>> REQUEST  POST https://yourorg.crm.dynamics.com/api/data/v9.2/accounts
    Authorization: [REDACTED]
    Accept: application/json
    Content-Type: application/json
    OData-MaxVersion: 4.0
    OData-Version: 4.0
    User-Agent: DataverseSvcPythonClient:0.1.0b8
    x-ms-client-request-id: 7050c4d0-6bcc-48e3-a310-b4e8fa18ac69
    x-ms-correlation-id: 4cace77d-e4ee-4419-8c65-fc62beed6e71
    Body:    {"name":"Contoso Ltd"}
[2026-04-11T15:27:31-0700] DEBUG <<< RESPONSE 204 POST https://yourorg.crm.dynamics.com/api/data/v9.2/accounts (78.0ms)
    Content-Type: application/json; odata.metadata=minimal
    OData-Version: 4.0
    x-ms-service-request-id: a6d0b6c4-5dd1-47cb-83eb-b6fccf754216
    x-ms-ratelimit-burst-remaining-xrm-requests: 7998
```

> **Security note:** This feature is intended for development and debugging only.
> Log files are **plaintext** and may contain PII, sensitive business data, and
> Dataverse record IDs — even with `max_body_bytes=0` (the default), request URLs
> can include filter values and record identifiers.
>
> - **Never enable in production.** If required for production diagnostics, keep
>   `max_body_bytes=0` and treat log files as regulated data under your organization's
>   data handling policy.
> - **Restrict access.** Set file system permissions so only the process user can
>   read log files. Use an encrypted volume or folder in sensitive environments.
> - **Control retention.** Log rotation keeps up to 5 files by default (`backup_count`).
>   Delete logs after the debugging session; use secure deletion for regulated data.
> - **Prevent source control leaks.** Add the log folder to `.gitignore` immediately.

### Limitations

- SQL queries are **read-only** and support a limited subset of SQL syntax
- Create Table supports the following column types: string, memo, int, decimal, float, bool, datetime, file, and picklist (Enum subclass)
- File uploads are limited by Dataverse file size restrictions (default 128MB per file)

## Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit [Contributor License Agreements](https://cla.opensource.microsoft.com).

When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

### API Design Guidelines

When contributing new features to this SDK, please follow these guidelines:

1. **Public methods in operation namespaces** - New public methods go in the appropriate namespace module under [operations/](src/PowerPlatform/Dataverse/operations/). Public types and constants live in their own modules (e.g., `models/metadata.py`, `common/constants.py`)
2. **Add README example for public methods** - Add usage examples to this README for public API methods
3. **Document public APIs** - Include Sphinx-style docstrings with parameter descriptions and examples for all public methods
4. **Update documentation** when adding features - Keep README and SKILL files (note that each skill has 2 copies) in sync
5. **Internal vs public naming** - Modules, files, and functions not meant to be part of the public API must use a `_` prefix (e.g., `_odata.py`, `_relationships.py`). Files without the prefix (e.g., `constants.py`, `metadata.py`) are public and importable by SDK consumers

## Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
