Composable data + AI systems

LLMs and data
Author

Cody Peterson

Published

October 15, 2023

Introduction

When writing (data and AI) code, it’s important to use the right abstraction for your job to be done. A modular and composable data and AI system is easier than ever with the new generation of LLM-powered tools, but lacks a standard, modular, interoperable, customizable, and extensible framework. Can Ibis Birdbrain be that standard?

Standards

When things don’t work as they should, it often means that standards are absent.

- The International Organization for Standardization (ISO)

- The Composable Codex

The standard that has emerged from the recent LLM applications is the power of a conversation in interacting with a computer. If we can give that interaction tht ability to interact with data platforms, we can achieve a lot through natural language.

Composable data (and AI) systems are MICE

Composable data systems are MICE: modular, interoperable, customizable, and extensible.

With any new tech, there’s a lot of duplication of effort and reinvention of wheels. Ibis Birdbrain aims to be an open-source standard for building data and AI systems in Python that adheres to MICE best practices and works with any data and AI platform. It achieves this by building on top of Ibis and Marvin, handling calls to the data and AI platforms respectively. Ibis and Marvin are themselves built on many other open-source projects, but provide the right level of abstraction for building composable data and AI systems, independently. Ibis Birdbrain glues them together enables a natural language interface that’s interoperable between 18+ data platforms and any AI platform Marvin supports.

glue that enables working via natural language with any data platform Ibis supports and any AI platform Marvin supports.

Important

Ibis Birdbrain is very early in development and looking for community input and contributors. We believe it’s important for there to be a standard, and I can see Ibis Birdbrain going in several directions:

  1. A useful template for building a language user interface with data tools
  2. A standard for building data and AI systems with MICE components

If you’re interested in either of these directions, please get involved with the project!

Ibis Birdbrain is under Ibis project governance.

Looking ahead

I am not a software engineer, I just saw a cool application of Ibis and Marvin. The objective of releasing Ibis Birdbrain as an open-source project under Ibis governance is to enable the community to build a standard for composable data and AI systems. My hope is for the Ibis, Marvin, and broader data/AI Python communities to build the standard in the open, and properietary applications on top of those standards. This is important for a composable data ecosystem that can be modular, interoperable, customizable, and extensible.

How is Ibis Birdbrain MICE now? What’s next?

Next steps

You can get involved with Ibis Birdbrain, our open-source data & AI project for building next-generation natural language interfaces to data.

Read the next post in this series, building Ibis Birdbrain.

Back to top