Metadata-Version: 2.1
Name: echoai-transformer-block
Version: 0.2
Summary: A package for Transformer blocks
Home-page: https://github.com/pratyakshagarwal/Echoai
Author: Echoai (pratyaksh agarwal)
Author-email: pratyakshagarwal93@gmail.com
License: MIT
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Description-Content-Type: text/markdown
Requires-Dist: torch

# Echoai Transformer Block Package

This package provides components for building Transformer blocks, including Multi-Head Attention and FeedForward layers.

## Installation

You can install the package using pip:

```bash
pip install echoai-transformer-block
```

Usage
Here's an example of how to use the `Block` class from this package:

import torch
from echoai_transformer_block import Block

# Define parameters
n_heads = 8
n_embed = 512
block_size = 16
dropout = 0.1
expand = 4

# Create a Block instance
block = Block(n_heads, n_embed, block_size, dropout, expand)

# Example input tensor
input_tensor = torch.randn(1, 16, 512)

# Forward pass through the Block
output_tensor = block(input_tensor)

print("Output shape:", output_tensor.shape)

### Components
`Block`

from echoai_transformer_block import Block

The `Block` class represents a Transformer block containing a Multi-Head Attention layer followed by a FeedForward layer.

`MultiAttentionHead`
from echoai_transformer_block import MultiAttentionHead
The `MultiAttentionHead` class represents the Multi-Head Attention mechanism used in Transformers.

`FeedForward`
from echoai_transformer_block import FeedForward
The `FeedForward` class represents the FeedForward module used in Transformer encoder layers.


Requirements
Python 3.7+
PyTorch

License
This project is licensed under the MIT License - see the LICENSE file for details.

This README file includes:
- The updated package name "Echoai Transformer Block".
- Installation instructions for the package.
- An example of how to use the `Block` class.
- Descriptions and import statements for each component (`Block`, `MultiAttentionHead`, `FeedForward`).
- Requirements section listing the required Python version and PyTorch.
- Mention of the license for the package.

You can include this updated `README.md` file in the root directory of your package alongside other files. When users visit your package's repository on GitHub or PyPI, they will see this README file, providing them with information on how to use your package.
