Metadata-Version: 2.4
Name: tcorpus
Version: 0.1.6
Summary: A powerful CLI-based text corpus analyser for extracting palindromes, anagrams, word frequencies, pattern matches, emails, and phone numbers from text files or direct input.
Author-email: Raghu <raghu59770@gmail.com>
Maintainer: Santosh, Raghuram
License: MIT
Project-URL: Homepage, https://github.com/yourusername/text-corpus-analyser
Project-URL: Documentation, https://github.com/yourusername/text-corpus-analyser#readme
Project-URL: Repository, https://github.com/yourusername/text-corpus-analyser
Project-URL: Issues, https://github.com/yourusername/text-corpus-analyser/issues
Keywords: text-analysis,corpus,nlp,cli,palindrome,anagram,word-frequency,pattern-matching,email-extraction,phone-extraction
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# tcorpus - Text Corpus Analyser

A powerful command-line tool for analyzing text corpora. Extract palindromes, anagrams, word frequencies, pattern matches, emails, and phone numbers from text files or direct input.

## Features

- 🔍 **Word Analysis**: Find palindromes, anagrams, and word frequencies
- 🎯 **Pattern Matching**: Search words using wildcard patterns and masks
- 📧 **Email Extraction**: Extract email addresses from text
- 📱 **Phone Number Detection**: Find phone numbers in various formats
- 🚫 **Stopword Filtering**: Filter out common words using config files or CLI options
- 📊 **Multiple Output Formats**: Save results as JSON or CSV
- ⚡ **Fast & Lightweight**: Zero external dependencies, pure Python standard library

## Installation

### From PyPI (Recommended)

```bash
pip install tcorpus
```

### From Source

```bash
git clone <repository-url>
cd text-corpus-analyser
pip install -e .
```

## Requirements

- Python 3.8 or higher
- No external dependencies (uses only Python standard library)

## Quick Start

```bash
# Get help
tcorpus --help

# Analyze a text file
tcorpus all demo.txt output.json

# Use direct text input
tcorpus palindrome --text "A man a plan a canal Panama" output.json
```

## Usage

### Input Options

You can provide input in two ways:

1. **From a file**:
   ```bash
   tcorpus <command> <input_file> <output_file>
   ```

2. **Direct text** (using `-t` or `--text`):
   ```bash
   tcorpus <command> --text "Your text here" <output_file>
   ```

### Common Options

All commands support these filtering options:

- `--stopwords <word1> <word2> ...` - Additional stopwords to ignore (combined with config.ini)
- `--starts-with <letter>` - Keep only words starting with this letter
- `--config <path>` - Path to config file for stopwords (default: `config.ini`)
- `-pw, --print-words` - Print filtered results to terminal

### Config File

Create a `config.ini` file in your working directory to define default stopwords:

```ini
[stopwords]
words = the,or,but,a,an,is,are,was,were
```

---

## Commands

### 1. Palindrome Detection

Find all palindromes (words that read the same forwards and backwards) in your text.

**Syntax:**
```bash
tcorpus palindrome [input_file] <output_file> [options]
```

**Examples:**
```bash
# From file
tcorpus palindrome demo.txt palindromes.json

# Direct text
tcorpus palindrome --text "madam level cat radar" output.json

# With filters
tcorpus palindrome demo.txt output.json --starts-with m --print-words

# Filter out stopwords
tcorpus palindrome demo.txt output.json --stopwords the a an
```

**Output Format:**
```json
{
  "palindromes": ["level", "madam", "radar"]
}
```

---

### 2. Anagram Detection

Find groups of words that are anagrams of each other.

**Syntax:**
```bash
tcorpus anagram [input_file] <output_file> [options]
```

**Examples:**
```bash
# Basic usage
tcorpus anagram demo.txt anagrams.json

# With direct text
tcorpus anagram --text "cat act tac dog god" output.json

# Print results to terminal
tcorpus anagram demo.txt output.json --print-words
```

**Output Format:**
```json
{
  "anagrams": [
    ["act", "cat", "tac"],
    ["dog", "god"]
  ]
}
```

---

### 3. Word Frequency Analysis

Count how often each word appears in the text.

**Syntax:**
```bash
tcorpus freq [input_file] <output_file> [options]
```

**Options:**
- `-w, --words <word1> <word2> ...` - Count specific words only, or use `all` for all words

**Examples:**
```bash
# Count all words
tcorpus freq demo.txt frequencies.json

# Count specific words
tcorpus freq demo.txt output.json --words python code program

# Count all words (explicit)
tcorpus freq demo.txt output.json --words all

# Output as CSV
tcorpus freq demo.txt frequencies.csv

# With filters
tcorpus freq demo.txt output.json --starts-with p --print-words
```

**Output Format (JSON):**
```json
{
  "frequencies": {
    "python": 5,
    "code": 3,
    "program": 2
  }
}
```

**Output Format (CSV):**
```csv
word,count
code,3
program,2
python,5
```

---

### 4. Pattern Matching (Mask)

Find words matching a specific pattern using wildcards and special syntax.

**Syntax:**
```bash
tcorpus mask <pattern> [input_file] <output_file> [options]
```

**Pattern Syntax:**
- `*` - Matches zero or more characters (wildcard)
- `?` - Matches exactly one character
- `word+` - Words starting with "word" (e.g., `ram+` matches "ram", "ramesh", "ramadan")
- `+word` - Words ending with "word" (e.g., `+ing` matches "running", "coding")
- `+word+` - Words containing "word" (e.g., `+ram+` matches "program", "ramesh")

**Options:**
- `--min-length <n>` - Minimum word length
- `--max-length <n>` - Maximum word length
- `--length <n>` - Exact word length
- `--contains <substring>` - Word must contain this substring

**Examples:**
```bash
# Wildcard pattern: words starting with 's' and ending with 'e'
tcorpus mask "s*e" demo.txt output.json

# Single character wildcard
tcorpus mask "c?t" demo.txt output.json

# Starts with pattern
tcorpus mask "ram+" demo.txt output.json

# Ends with pattern
tcorpus mask "+ing" demo.txt output.json

# Contains pattern
tcorpus mask "+ram+" demo.txt output.json

# With length filters
tcorpus mask "s*e" demo.txt output.json --min-length 4 --max-length 6

# Exact length
tcorpus mask "a*d" demo.txt output.json --length 5

# Must contain substring
tcorpus mask "s*e" demo.txt output.json --contains "al"

# Print matches to terminal
tcorpus mask "s*e" demo.txt output.json --print-words
```

**Output Format:**
```json
{
  "mask_matches": ["sale", "safe", "same", "sane", "site", "size"]
}
```

---

### 5. Email Extraction

Extract email addresses from text.

**Syntax:**
```bash
tcorpus email [input_file] <output_file> [options]
```

**Examples:**
```bash
# From file
tcorpus email demo.txt emails.json

# Direct text
tcorpus email --text "Contact us at info@example.com or support@test.org" output.json

# Print to terminal
tcorpus email demo.txt output.json --print-words
```

**Output Format:**
```json
{
  "emails": ["info@example.com", "support@test.org"]
}
```

---

### 6. Phone Number Extraction

Extract phone numbers from text in various formats.

**Syntax:**
```bash
tcorpus phone [input_file] <output_file> [options]
```

**Options:**
- `-d, --digits <n>` - Minimum number of digits required (default: 10)

**Examples:**
```bash
# Basic usage (default: 10 digits minimum)
tcorpus phone demo.txt phones.json

# Custom minimum digits
tcorpus phone demo.txt output.json --digits 7

# Direct text
tcorpus phone --text "Call +1 (555) 123-4567 or 07123 456789" output.json

# Print to terminal
tcorpus phone demo.txt output.json --print-words
```

**Output Format:**
```json
{
  "phone_numbers": ["+1 (555) 123-4567", "07123 456789"]
}
```

---

### 7. All Analyses

Run all analyses at once: palindromes, anagrams, frequencies, emails, and phone numbers.

**Syntax:**
```bash
tcorpus all [input_file] <output_file> [options]
```

**Options:**
- `-w, --words <word1> <word2> ...` - Words to count (use `all` for all words)
- `-d, --digits <n>` - Minimum digits for phone numbers (default: 10)

**Examples:**
```bash
# Run all analyses
tcorpus all demo.txt complete_analysis.json

# With word frequency filter
tcorpus all demo.txt output.json --words python code

# Custom phone digit requirement
tcorpus all demo.txt output.json --digits 7

# With text filters
tcorpus all demo.txt output.json --starts-with a --stopwords the a an

# Print all results to terminal
tcorpus all demo.txt output.json --print-words
```

**Output Format:**
```json
{
  "palindromes": ["level", "madam"],
  "anagrams": [["act", "cat", "tac"]],
  "frequencies": {
    "python": 5,
    "code": 3
  },
  "emails": ["info@example.com"],
  "phone_numbers": ["+1 (555) 123-4567"]
}
```

---

## Advanced Examples

### Combining Filters

```bash
# Find palindromes starting with 'm', excluding stopwords
tcorpus palindrome demo.txt output.json \
  --starts-with m \
  --stopwords the a an \
  --print-words
```

### Processing Compressed Files

The tool automatically handles `.gz` compressed files:

```bash
tcorpus all large_text.txt.gz output.json
```

### Batch Processing

```bash
# Process multiple files
for file in *.txt; do
  tcorpus all "$file" "${file%.txt}_analysis.json"
done
```

---

## Output Formats

### JSON (Default)

All commands output JSON by default. The structure varies by command:

- **Single analysis**: `{"palindromes": [...]}`
- **All analyses**: `{"palindromes": [...], "anagrams": [...], ...}`

### CSV (Frequency Only)

When using `freq` command with `.csv` extension:

```bash
tcorpus freq demo.txt frequencies.csv
```

Outputs:
```csv
word,count
python,5
code,3
```

---

## Troubleshooting

### Command Not Found

If `tcorpus` is not recognized after installation:

1. **Check installation:**
   ```bash
   pip show tcorpus
   ```

2. **Verify PATH** includes Python Scripts directory:
   ```bash
   python -m site --user-base
   ```

3. **Use module execution:**
   ```bash
   python -m tcorpus --help
   ```

### Config File Not Found

If `config.ini` is missing, the tool will still work but won't use default stopwords. Create a `config.ini` file in your working directory:

```ini
[stopwords]
words = the,or,but,a,an,is,are
```

---

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Authors

- **Raghu** - raghu59770@gmail.com

## Maintainers

- Santosh
- Raghuram

## License

This project is open source and available for use.

## Version

Current version: 0.1.5

---

## Support

For issues, questions, or contributions, please open an issue on the project repository.



