Metadata-Version: 2.3
Name: es-testbed
Version: 0.11.2
Summary: Library to help with building and tearing down indices, data streams, repositories and snapshots
Project-URL: Documentation, https://github.com/untergeek/es-testbed#readme
Project-URL: Issues, https://github.com/untergeek/es-testbed/issues
Project-URL: Source, https://github.com/untergeek/es-testbed
Author-email: Elastic <info@elastic.co>
License: Apache-2.0
License-File: LICENSE
Keywords: datastream,elasticsearch,index,repository,snapshot,testing
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Requires-Dist: dotmap==1.3.30
Requires-Dist: es-client>=8.18.1
Requires-Dist: es-wait>=0.15.0
Requires-Dist: gitpython==3.1.43
Requires-Dist: tiered-debug==1.2.1
Provides-Extra: doc
Requires-Dist: sphinx; extra == 'doc'
Requires-Dist: sphinx-rtd-theme; extra == 'doc'
Provides-Extra: test
Requires-Dist: mock; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Requires-Dist: pytest-dotenv; extra == 'test'
Requires-Dist: pytest>=7.2.1; extra == 'test'
Requires-Dist: requests; extra == 'test'
Description-Content-Type: text/markdown

# es-testbed
A way to create indices, datastreams, and snapshots to facilitate testing.

# Preliminary Documentation

## 1. Create a Preset

Create a preset directory. An example preset directory is in
`src/es_testbed/presets/searchable_test`.

Your preset directory must include the following files:

- A plan YAML file, e.g. `plan.yml`
- A buildlist YAML file, e.g. `buildlist.yml`
- A `functions.py` file (the actual python code), which must contain a
  function named `doc_generator()`. This function must accept all kwargs from
  the buildlist's `options`
- A `definitions.py` file, which is a Python variable file that helps find
  the path to the module, etc., as well as import the plan, the buildlist,
  the mappings and settings, etc. This must at least include a `get_plan()`
  function that returns a dictionary of a built/configured plan.
- A `mappings.json` file (contains the index mappings your docs need)
- A `settings.json` file (contains the index settings)
Any other files can be included to help your doc_generator function, e.g.
Faker definitions and classes, etc. Once the preset module is imported,
relative imports should work.

**Note:** If `ilm['enabled'] == False`, the other subkeys will be ignored. In fact, `ilm: False` is also acceptable.

Acceptable values for `readonly` are the tier names where `readonly` is acceptable: `hot`, `warm`, or `cold`.

Save this for step 2.

## 2. Create a TestBed

**Must have an Elasticsearch client established to proceed**

```
from es_testbed import TestBed

tb = TestBed(client, **kwargs)
```

For a builtin preset, like `searchable_test`, this is:

```
tb = TestBed(client, builtin='searchable_test', scenario=None)
```

`scenario` can be one of many, if configured.

For using your own preset in a filesystem path:

```
tb = TestBed(client, path='/path/to/preset/dir')
```

`path` _must_ be a directory.

For importing from a Git repository:

```
tb = TestBed(client, url='user:token@https://github.com/GITUSER/reponame.git', ref='main', path='subpath/to/preset')
```

Note that `user:token@` is only necessary if it's a protected repository.
In this case `path` must be a subdirectory of the repository. `depth=1` is manually set, so only the most recent
bits will be pulled into a tmpdir, which will be destroyed as part of `teardown()`.

### 2.1 Index Template creation (behind the scenes)

Based on the settings in step 1, you will have 2 component templates and one index template that
references them:

#### Component Template 1

```
es-testbed-cmp-mytest-000001
```

Settings:

```
{'index.number_of_replicas': 0}
```

If using a `rollover_alias` or ILM Policy, then additional values will automatically be added.

#### Component Template 2

```
es-testbed-cmp-mytest-000002
```

Mappings:

```
{
    'properties': {
        '@timestamp': {'type': 'date'},
        'message': {'type': 'keyword'},
        'number': {'type': 'long'},
        'nested': {'properties': {'key': {'type': 'keyword'}}},
        'deep': {'properties': {'l1': {'properties': {'l2': {
            'properties': {'l3': {'type': 'keyword'}}}}}}
        }
    }
}
```

#### Index Template
```
es-testbed-tmpl-mytest-000001
```

### 2.2 You have indices

Based on what was provided in step 1, you will have 3 indices with a basic mapping, and 10 documents each:

```
es-testbed-idx-mytest-000001
es-testbed-idx-mytest-000002
es-testbed-idx-mytest-000003
```

Documents will have been added per the TestPlan settings. The orthography for these documents is in
`es_testbed.helpers.utils.doc_gen()`. Counts are preserved and continue to grow from one index to
the next.

## 3. Perform your tests.

This is where the testing can be performed.

## 4. Teardown

```
tb.teardown()
```

Barring anything unusual happening, all indices, data_streams, ILM policies, index & component
templates, and snapshots (if an index is promoted to searchable snapshots) will be deleted as part
of the `teardown()` method.