Cortiqa Falin-300M (Preview)

Falin-300M is a 297-Million parameter decoder-only transformer model designed, engineered, and trained from scratch by Cortiqa. Built upon our proprietary Menothus architecture, Falin is optimized for extreme low-latency and edge device deployment (consumer GPUs, CPUs, mobile devices, and browser extensions).

This model serves as a core Proof of Concept (PoC) and validation milestone demonstrating our ability to architect, train, and deploy sovereign LLMs and SLMs completely in-house.

Note: This is a base / text-completion model, not an instruction-tuned chat assistant. It is designed to be fine-tuned for downstream tasks.

Model Architecture & Innovations

• •
  Total Parameters:297,034,800 (~300M parameters).
• •
  Extreme Grouped Query Attention (GQA):16 Query heads to 2 Key-Value heads (8:1 ratio), reducing KV-cache VRAM consumption by 75% during inference.
• •
  Parallel Attention + SwiGLU FFN:Computes Attention and Feed-Forward networks concurrently, improving GPU utilization and decreasing per-layer execution latency.
• •
  Hybrid Sliding Window Attention (SWA):Local context window of 512 tokens with every 4th layer computing dense global causal attention.
• •
  FlashAttention / SDPA Native:Fully optimized for scaled dot product attention.

Technical Specifications

Training Data

Pre-trained from scratch on a curated dataset (~40M tokens), including general text and a dedicated identity dataset so the model self-identifies as “Falin.” This is an early-stage dataset — scale and diversity are actively being expanded in future releases.

Limitations

• •
  This is an Alpha research preview — not production-ready.
• •
  As a 297M base model trained on a relatively small dataset, it may produce incoherent or hallucinated output on general/open-domain prompts, especially outside its training distribution.
• •
  Not instruction-tuned — it will not behave like a chat assistant out of the box. Use it as a base for fine-tuning.
• •
  Max context length is limited to 1024 tokens.

How to Run Inference

1. Install requirements:

pip install torch tokenizers huggingface_hub

2. Python inference:

import torch
from huggingface_hub import snapshot_download

# Download model repository from Hugging Face
model_dir = snapshot_download(repo_id="Cortiqa/Falin-300M-Preview")

# Load model weights and config using Menothus architecture code from repo

Fine-Tuning

Falin-300M is released specifically to be fine-tuned — it is not meant to be used directly for chat or instruction-following. Because it's only ~300M parameters, it's cheap and fast to fine-tune, even on a single consumer GPU or a free-tier Colab instance.

What you can fine-tune it for:

• •
  Domain-specific text generation (e.g. legal, medical, customer support in your niche).
• •
  Instruction-following / chat behavior (using an instruction dataset + SFT).
• •
  Regional language or dialect adaptation.
• •
  Task-specific completion (classification-as-generation, summarization, etc.).

Push your fine-tuned checkpoint back to the Hub tagged with base_model: Cortiqa/Falin-300M-Preview so it shows up under this model's Finetunes tree.

Evaluation

Formal benchmarks (MMLU, ARC, etc.) are planned for a future release as training data and scale increase.

About Cortiqa & License

Falin-300M was designed, engineered, and pre-trained from scratch by Cortiqa, focusing on building sovereign, ultra-fast, and resource-efficient AI architectures for India and the global developer ecosystem.

This model is released under the Cortiqa Falin Non-Commercial License. Free for research, evaluation, and non-commercial fine-tuning. Commercial use requires written permission — contact team@cortiqa.co.