Execution Environments

Relevant source files

• notebook/analytics_user_acquisition.ipynb
• notebook/feature/basic_building_blocks.ipynb
• notebook/feature/recency_embedding.ipynb
• notebook/rag_hr_knowledgebase.ipynb
• notebook/recommendations_e_commerce.ipynb
• notebook/semantic_search_netflix_titles.ipynb
• notebook/semantic_search_news.ipynb

This document describes the different execution environments available in Superlinked for running vector search applications. Execution environments determine how your application processes data, handles queries, and manages vector storage during development, testing, and production phases.

For information about specific vector database integrations, see Vector Databases. For production deployment configuration, see Production Setup.

Overview

Superlinked provides three primary execution environments, each designed for different stages of development and deployment:

Environment	Primary Use Case	Data Persistence	Performance	Setup Complexity
In-Memory	Development, prototyping, testing	Temporary	Fast for small datasets	Minimal
Interactive	Jupyter notebooks, experimentation	Session-based	Moderate	Low
REST API	Production deployment	External VDB	High, scalable	High

Execution Environment Architecture

Sources: notebook/feature/basic_building_blocks.ipynb141-143 notebook/recommendations_e_commerce.ipynb374-376 notebook/semantic_search_netflix_titles.ipynb374-376

Executor Class Hierarchy

Sources: notebook/feature/basic_building_blocks.ipynb141-142 notebook/recommendations_e_commerce.ipynb373-374

Environment Selection Criteria

Development Environment (InMemoryExecutor)

The in-memory environment is ideal for:

• Initial prototyping and experimentation
• Unit testing and validation
• Small dataset exploration
• Schema and space configuration development

Key characteristics:

• Data stored in Python memory
• No external dependencies
• Fast iteration cycles
• Automatic cleanup on process termination

executor = sl.InMemoryExecutor(sources=[source], indices=[paragraph_index])
app = executor.run()

Sources: notebook/feature/basic_building_blocks.ipynb142-143

Interactive Environment

Interactive execution extends in-memory capabilities for:

• Jupyter notebook workflows
• Data science experimentation
• Educational demonstrations
• Iterative query refinement

Enhanced features include:

• Session persistence within notebook context
• Visualization integration with Altair
• DataFrame conversion utilities
• Dynamic parameter adjustment

Sources: notebook/recommendations_e_commerce.ipynb84-87 notebook/semantic_search_netflix_titles.ipynb74-78

Production Environment (RestExecutor)

Production deployment requires:

• Persistent vector storage
• Horizontal scalability
• High availability
• RESTful API interface
• Multi-client support

Production components:

• External vector databases for persistence
• HTTP server for client communication
• Batch data ingestion capabilities
• Monitoring and telemetry integration

Sources: Based on architecture diagrams showing production deployment patterns

Data Sources by Environment

Source Type	Development	Interactive	Production
InMemorySource	✓ Primary	✓ Primary	-
RestSource	-	-	✓ Primary
DataLoaderSource	-	-	✓ Batch
DataFrame input	✓	✓	-
Dictionary input	✓	✓	-

Context Data Management

All execution environments support context data for temporal operations:

EXECUTOR_DATA = {sl.CONTEXT_COMMON: {sl.CONTEXT_COMMON_NOW: timestamp}}
executor = sl.InMemoryExecutor(
    sources=[source], 
    indices=[index], 
    context_data=EXECUTOR_DATA
)

This enables:

• Recency space calculations
• Temporal filtering
• Time-aware recommendations
• Event effect processing

Sources: notebook/semantic_search_netflix_titles.ipynb93 notebook/rag_hr_knowledgebase.ipynb101 notebook/analytics_user_acquisition.ipynb102

Query Execution Patterns

Different environments support varying query patterns:

Synchronous Queries (Development/Interactive)

result = app.query(query, query_text="search term")
pandas_df = sl.PandasConverter.to_pandas(result)

Asynchronous Queries (Production)

• HTTP POST requests to REST endpoints
• JSON response format
• Concurrent query processing
• Load balancing support

Sources: notebook/feature/basic_building_blocks.ipynb238-240