Metadata-Version: 2.4
Name: llnl-sina
Version: 1.16.1
Summary: Simulation INsight and Analysis
Author: Siboka Team
Author-email: weave-support@llnl.gov
License: MIT
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
License-File: LICENSE
Requires-Dist: six
Requires-Dist: sqlalchemy
Requires-Dist: sqlalchemy<2; python_version < "3"
Requires-Dist: enum34; python_version < "3.4"
Requires-Dist: orjson; python_version >= "3.6" and platform_machine != "ppc64le"
Requires-Dist: ujson; python_version >= "3.6" and platform_machine == "ppc64le"
Requires-Dist: ujson<4; python_version < "3.6" and platform_machine != "ppc64le"
Requires-Dist: freetype-py; platform_system == "Darwin"
Requires-Dist: pandas
Requires-Dist: h5py
Requires-Dist: deepdiff
Requires-Dist: texttable
Provides-Extra: cassandra
Requires-Dist: cassandra-driver; extra == "cassandra"
Provides-Extra: jupyter
Requires-Dist: jupyter-client; extra == "jupyter"
Requires-Dist: ipython<8.13; extra == "jupyter"
Requires-Dist: ipykernel; extra == "jupyter"
Requires-Dist: ipywidgets; extra == "jupyter"
Requires-Dist: matplotlib; extra == "jupyter"
Requires-Dist: tabulate; extra == "jupyter"
Requires-Dist: ipympl; extra == "jupyter"
Provides-Extra: mysql
Requires-Dist: mysql-connector-python; extra == "mysql"
Provides-Extra: development
Requires-Dist: pytest; extra == "development"
Requires-Dist: pytest-cov; extra == "development"
Requires-Dist: flake8; extra == "development"
Requires-Dist: mock; extra == "development"
Requires-Dist: sphinx; extra == "development"
Requires-Dist: sphinx_rtd_theme; extra == "development"
Requires-Dist: sphinx_immaterial; extra == "development"
Requires-Dist: myst-nb; extra == "development"
Requires-Dist: nbformat; extra == "development"
Requires-Dist: nbconvert; extra == "development"
Requires-Dist: ipykernel; extra == "development"
Requires-Dist: black; extra == "development"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: summary

Contents
========
- Overview
- LC Setup
- Standard Setup
- Manual Setup

    - Creating the Environment
    - Installing Software Dependencies

- Activating and Deactivating the Virtual Environment
- Using the Example Notebooks and Datasets
- Testing
- Supported Environments
- Database Support


Overview
========

Sina's Python component is a tool for making simulation (meta)data collection
and exploration simple.

It works by collecting information from code runs, logs, and other outputs into
a common file format which can then be passed off to one of Sina's supported
backends, all of which are queried using the same user-friendly Python API. To the
end user, this means that important data can be accessed through Python scripts,
GUIs (Jupyter notebooks) etc. with all the speed of a database and none of the
complexity (the user never has to interact with database architecture), nor any of
the traditional headaches of parsing logs or remembering which file contains what.

Sina is integrated into a number of LLNL physics codes to capture simulation data;
look for the _sina.json! If your code isn't configured to output Sina, but you'd
like it to be, we may be able to work with the code team to integrate it--you can reach
us at weave-support@llnl.gov, or check out the WEAVE project on Gitlab and Teams.

The instructions below will guide you through setting up a virtual environment for
Sina (or installing it in one that already exists), running example notebooks, and
getting dependencies for your backend(s) of choice. Note that SQL
will always be available as the "default" backend. Once you're done with setup,
a quickstart tutorial can be found in notebook form at
<sina_root>/examples/basic_usage.ipynb.

Remember that, if you're on LC, each time you log in you'll first need to activate the
environment. When you're done, we recommend you deactivate the virtual environment to get
back to your default environment or end your session.


LC Setup
========

If you're on an LC machine, you can use a virtual environment with dependencies
already installed::

    $ source /collab/usr/gapps/wf/releases/sina/bin/activate

The above is for bash; other activation scripts, e.g. activate.csh, can be found
in the same directory.

Sina will now be available for use via Python virtual environment, and can be
tested with `sina -h` (which should display a help message). When you're done,
use `deactivate` to exit the virtual environment. Note that this will be the release
(master) Sina version--if you want to use Sina Develop, keep reading!

If you run into issues with the LC virtual environment, please email us at weave-support@llnl.gov.


Standard Non-LC Setup
=====================

Sina is available on PyPi::

    $ pip install llnl-sina

However, this will only give you access to the release version! Non-release
versions are not available externally. Internal users looking to use our development
version, or wanting to contribute to Sina, clone us from CZ Gitlab. External
contributors should clone us from the LLNL Github.

After cloning, install like any other standard python lib (cd to here, `pip install .`).

Notably, Sina contains a number of modules:
   - jupyter adds support for jupyter notebook visualizations
   - mysql adds support for using mysql and mariadb
   - cassandra adds limited support for Cassandra (NoSQL), but is deprecated
   - development adds dependencies for our test/CI

If you want any of the above, add them to your install command like::

   $ pip install .[jupyter,development]

LC users have all optional dependencies available through the sina venv.


Using the Example Notebooks and Datasets
========================================

Sina contains tutorials in the form of Jupyter notebooks.
Files are stored in the examples directory (found in the
sina root folder alongside the python and cpp folders), and are organized by
dataset, with data_overview.rst containing descriptions of each set.
To use the notebooks, you'll first need to run getting_started.ipynb
(also in the examples directory) from the LC Jupyter server at
lc.llnl.gov/jupyter. This will create a Jupyter kernel from your current virtual
environment, making anything installed in it available to the notebook.
After that, you'll be ready to run the rest of the
notebooks. If you're not working on LC, you can also set Jupyter up locally:
run `make Jupyter` from the python folder, then `jupyter notebook`. This will
open a webpage similar to what you'd see accessing LC's Jupyter server.

Most notebooks rely on sample datasets. Pre-built sets are deployed
with Sina to the LC, but you can build them locally as well to experiment with
Sina. Go into any dataset folder (the NOAA set is well-sized for experimentation)
and `./build_db.sh`. Note that you'll need Sina available to do so, see the
section on virtual environments.

To clean all output from the notebooks::

    (venv) $ make clean-notebooks


Testing
=======

Sina uses gitlab CI to test out MRs. To run a local equivalent, you can
use pylint and the tests found in the `scripts` folder. For example::

    $ python3 -m venv sample_venv && source sample_venv/bin/activate
    $ pip install -e.[development]
    $ pytest -v -m "not cassandra and not mysql"
    $ ../scripts/test_*

This will install all necessary dependencies and run most tests.
Excluded test categories (ex: mysql) require additional setup
(ex: sacrificial database) and may be better left to the CI; if
you want example setups, see the .gitlab-ci.yml.

If you need a dependency added, email us so I can re-run the nightly
environment rebuild.


Supported Environments
======================

Sina is most regularly tested against Python 3.9 and Python 3.14 in a
RHEL4 environment.


Database Support
================

As mentioned above, "stock" Sina doesn't include dependencies for databases beyond
SQLite, but Sina supports and test against them. External users need to install them,
ex: `pip install .[mysql]`, and also need to have a reachable database.

LC venv users already have all additional modules installed, so specify a myql connector
string in place of a db name and you should be good to go. If you used LaunchIT
(lit.llnl.gov) to provision your MariaDB instance, they make this very easy to get, look
for the "Sina Connection String" on your dashboard and paste it in like so::

    $ sina ingest --database-type=sql --database "mysql+mysqlconnector://host:port/?read_default_file=~/.my.cnf"

LaunchIT also provides contents for the .my.cnf file, which you can paste to a location
of your choosing, with the above assuming you placed it in $HOME.

Here's how to use it with the API::

    datastore = sina.connect("mysql+mysqlconnector://host:port/?read_default_file=~/.my.cnf")
