⚡ Quick Start & CLI Reference

Once installed, the unified unfish binary provides commands across the entire software development lifecycle:

🛠️ Installation & Build Guide

Unfish requires only a standard C99 compiler (such as gcc or clang) and standard POSIX make. It has zero external dependencies beyond libc and -lm.

Building from Source

git clone https://github.com/ashokwebs/unfish.git
cd unfish
make -j4
./bin/unfish version

Running the Test Suites

# Run all 21 unit test suites and the 104-test differential conformance suite
make test

# Verify zero memory leaks under AddressSanitizer & UndefinedBehaviorSanitizer
make test-asan

# Run multi-tier performance benchmarks
make bench

🐡 What is Unfish?

Unfish is a real, general-purpose programming language, runtime, compiler, and educational computing environment written in clean ANSI C99 with zero external dependencies.

It is designed to provide an unbroken path of intellectual ascent: from visual block programming, into clean indentation-based text programming, to algorithms, data structures, runtime architecture, garbage collection, bytecode virtual machines, and low-level systems programming.

"Hide complexity until the learner is ready — never hide it forever."

Five Execution Engines in Lockstep

Unfish implements five distinct execution backends, kept in lockstep by a differential test suite that requires byte-identical output from every one of them:

1. Tree-Walking AST Interpreter: Immediate startup, direct AST evaluation, ideal for REPL and visual block editing.
2. Bytecode Virtual Machine (--vm): 59-opcode stack VM with lexical upvalue capture cells, exception unwinding, and 2x–6x speedup over AST tree-walking.
3. Register-Based Bytecode VM (--regvm): 256-register, 3-address VM with computed-goto dispatch.
4. Native C99 AOT Compiler (build): Direct transpilation into standalone ANSI C99 with single-header runtime, measured at 25–60x faster than AST.
5. WebAssembly Target (build --wasm): Portable WASM binary running in browsers and edge runtimes.

🚀 Your First Program

Unfish source code uses the .unfish or .uf extension. Here is a program demonstrating string interpolation, structs, and list transformations:

💻 REPL & Developer Toolchain

The unfish executable provides a unified interface across the development lifecycle:

🔬 Learn the Computer through Unfish

A defining mission of Unfish is to teach computer systems by making every layer of language execution visible and tangible. Type any code below and click through the tabs to observe how human-readable text transforms into tokens, syntax trees, bytecode instructions, and runtime memory.

01. Hello World & Text Output

In Unfish, printing to standard output is done with say (appends a newline) or the print(value) function (no trailing newline). Unfish also features Python-style string interpolation via f"...".

02. Values & Primitive Types

Unfish runtime values are represented internally by a 16-byte tagged union (UfValue). The primitive types are:

• Numbers — 64-bit IEEE 754 floating-point numbers (42, 3.14159, 0xFF, 1e6).
• Strings — Contiguous UTF-8 encoded text with length tracking and string interning.
• Booleans — true or false.
• Null — Represents absence of value (null).

03. Variables & Lexical Scope

Variables in Unfish are declared with the let keyword. Once declared, variables can be reassigned with =:

04. Expressions & Operators

Unfish parses expressions using Vaughan Pratt's top-down operator precedence table. It supports standard arithmetic (+, -, *, /, %), comparisons, logical operators (and, or, not), and the forward pipe operator (|>).

05. Conditional Logic

Blocks in Unfish are delimited using off-side indentation (4 spaces) introduced with a colon :.

06. Loops & Iteration

Unfish supports three loop constructs:

• while <condition>: — Loop while condition remains truthy.
• for item in <collection>: — Iterate over arrays, strings, or map keys.
• repeat <count> times: — Dedicated beginner-friendly fixed count loop.

07. Functions & Default Parameters

Functions are declared with function or fn and can include default parameter values:

08. Higher-Order Functions

Functions can be passed as arguments or returned from other functions. Unfish provides map, filter, reduce, sort, and zip out of the box:

09. Closures & Functional Pipelines

Functions in Unfish capture variables from their enclosing lexical environments. The pipe operator (|>) allows chaining function calls cleanly without nested parentheses.

10. Arrays, Maps & Byte Buffers

Unfish features dynamic lists, hash maps with string keys, and contiguous binary byte buffers for low-level systems programming:

11. Destructuring & Rest Patterns

Unpack values from arrays or maps into named variables using let [a, b, ...rest] = ...:

12. Structs & Object Methods

Unfish provides lightweight structs with instance fields and methods bound to self:

13. Enums & Pattern Matching

Enums in Unfish are tagged sum types whose variants can carry data, matched with match ... when ...:

14. Error Recovery (try / catch / finally)

Robust exception handling allows recovering from runtime errors or user-thrown exceptions with structured try ... catch err: ... finally: ...:

15. Memory Model & Garbage Collector

The Unfish runtime is designed for predictable low latency and tiny memory footprints:

• Tagged Unions (16 bytes): All stack values fit inside UfValue containing a 64-bit payload and an 8-bit type tag.
• Arena Allocators: Parsing AST nodes and compilation bytecode chunks use fast linear bump arenas freed in O(1) time.
• Mark-Sweep GC: Objects on the heap (strings, arrays, maps, instances, fibers) are managed by an exact tri-color mark-sweep garbage collector.

16. VM & Native Compilation

Unfish lets you choose your execution strategy depending on performance requirements:

17. Cooperative Fibers & CSP Channels

Concurrency in Unfish is based on lightweight green threads (fibers) and Communicating Sequential Processes (CSP) channels, operating with deterministic scheduling and zero race conditions.

18. Traits & Polymorphic Interfaces

Traits define abstract behavioral contracts that structs implement to enable polymorphic dispatch. Traits allow you to write generic algorithms that operate on any type providing required methods.

19. Pipelines & List Comprehensions

Unfish features point-free pipeline chaining via the |> operator and concise declarative transformations using list and map comprehensions.

20. Systems Programming: Byte Buffers & Bitwise Math

Unfish combines high-level syntax with direct low-level memory buffers (buffer) and 64-bit bitwise operators for wire-level networking and binary protocol framing.

21. Asynchronous Coroutines (Async / Await)

Asynchronous programming in Unfish is powered by async fn and await, desugaring to cooperative coroutine fibers and channel-based rendezvous under the centralized runtime scheduler.

22. Developer Tooling, Debugger & Language Server

Unfish is equipped with a rich, self-contained developer toolchain built directly into the unified unfish executable:

Core Built-in Functions

All core functions are available globally in every Unfish program without needing an import statement.

Module: sys

Access operating system parameters, environment variables, and process control. Import with import sys or from sys import platform, exit.

Module: fs (Sandboxed Filesystem)

File I/O and directory operations. Import with import fs.

Module: time

High-resolution wall clocks, timestamps, and delays. Import with import time.

Module: random

Pseudo-random number generation, shuffling, and seeding. Import with import random.

Module: json

Fast JSON parsing and serialization into Unfish maps and arrays. Import with import json.

Module: testing

Assertions and test suite framework. Run tests via unfish test.

📜 Formal Grammar & Language Specification

Unfish syntax is formally specified in Extended Backus-Naur Form (ISO/IEC 14977). The language utilizes lexical indentation (the off-side rule), structured expression pipelines, algebraic data types, and pattern matching.

Syntactic Grammar in ISO/IEC 14977 EBNF

(* Unfish Formal Language Grammar *)

Program
    = { Statement | TopLevelDecl } ;

TopLevelDecl
    = StructDecl
    | EnumDecl
    | ImportStmt
    | FunctionDecl ;

StructDecl
    = "struct" , Identifier , ":" , NEWLINE ,
      INDENT , { StructField | StructMethod } , DEDENT ;

StructField
    = Identifier , [ "=" , Expression ] , NEWLINE ;

StructMethod
    = ( "fn" | "function" ) , Identifier , "(" , [ ParameterList ] , ")" , ":" , Block ;

EnumDecl
    = "enum" , Identifier , ":" , NEWLINE ,
      INDENT , { VariantDecl } , DEDENT ;

VariantDecl
    = Identifier , [ "(" , IdentifierList , ")" ] , NEWLINE ;

ImportStmt
    = "import" , Identifier , [ "as" , Identifier ] , NEWLINE ;

FunctionDecl
    = ( "fn" | "function" ) , Identifier , "(" , [ ParameterList ] , ")" , ":" , Block ;

ParameterList
    = Parameter , { "," , Parameter } ;

Parameter
    = Identifier , [ "=" , Expression ] ;

Block
    = NEWLINE , INDENT , { Statement } , DEDENT ;

Statement
    = LetDecl
    | Assignment
    | IfStmt
    | WhileStmt
    | ForStmt
    | MatchStmt
    | TryCatchStmt
    | SayStmt
    | ReturnStmt
    | BreakStmt
    | ContinueStmt
    | ExpressionStmt ;

LetDecl
    = "let" , ( Identifier | DestructuringPattern ) , [ "=" , Expression ] , NEWLINE ;

DestructuringPattern
    = "[" , [ PatternList ] , "]"
    | "{" , [ MapPatternList ] , "}" ;

Assignment
    = PrimaryExpr , ( "=" | "+=" | "-=" | "*=" | "/=" | "%=" ) , Expression , NEWLINE ;

IfStmt
    = "if" , Expression , ":" , Block ,
      { "elif" , Expression , ":" , Block } ,
      [ "else" , ":" , Block ] ;

WhileStmt
    = "while" , Expression , ":" , Block ;

ForStmt
    = "for" , Identifier , "in" , Expression , ":" , Block ;

MatchStmt
    = "match" , Expression , ":" , NEWLINE ,
      INDENT , { MatchArm } , DEDENT ;

MatchArm
    = "when" , Pattern , ":" , Block ;

Pattern
    = Identifier , [ "(" , PatternList , ")" ]
    | Literal
    | "_" ;

TryCatchStmt
    = "try" , ":" , Block ,
      "catch" , [ Identifier ] , ":" , Block ,
      [ "finally" , ":" , Block ] ;

SayStmt
    = "say" , Expression , NEWLINE ;

ReturnStmt
    = "return" , [ Expression ] , NEWLINE ;

ExpressionStmt
    = Expression , NEWLINE ;

(* Expression Hierarchy with Pratt Precedence Climbing *)

Expression
    = PipelineExpr ;

PipelineExpr
    = LogicalOrExpr , { "|>" , ( Identifier | CallExpr ) } ;

LogicalOrExpr
    = LogicalAndExpr , { ( "or" | "||" ) , LogicalAndExpr } ;

LogicalAndExpr
    = EqualityExpr , { ( "and" | "&&" ) , EqualityExpr } ;

EqualityExpr
    = RelationalExpr , { ( "==" | "!=" ) , RelationalExpr } ;

RelationalExpr
    = AdditiveExpr , { ( "<" | "<=" | ">" | ">=" ) , AdditiveExpr } ;

AdditiveExpr
    = MultiplicativeExpr , { ( "+" | "-" ) , MultiplicativeExpr } ;

MultiplicativeExpr
    = UnaryExpr , { ( "*" | "/" | "%" ) , UnaryExpr } ;

UnaryExpr
    = ( "-" | "!" | "not" ) , UnaryExpr
    | PrimaryExpr ;

PrimaryExpr
    = Literal
    | Identifier
    | LambdaExpr
    | "(" , Expression , ")"
    | ArrayLiteral
    | MapLiteral
    | PrimaryExpr , "(" , [ ArgumentList ] , ")"
    | PrimaryExpr , "[" , Expression , "]"
    | PrimaryExpr , "." , Identifier ;

LambdaExpr
    = ( "fn" | "function" ) , "(" , [ ParameterList ] , ")" , ":" , ( Expression | Block ) ;

ArrayLiteral
    = "[" , [ Expression , { "," , Expression } ] , "]" ;

MapLiteral
    = "{" , [ MapEntry , { "," , MapEntry } ] , "}" ;

MapEntry
    = ( StringLiteral | Identifier ) , ":" , Expression ;

Pratt Operator Precedence & Associativity Table

⚙️ 59-Opcode Stack VM ISA Specification

The Unfish Virtual Machine is a bytecode-interpreted stack machine implemented in pure ANSI C99. It evaluates 16-byte tagged values (UfValue) on an operand stack with dynamic bounds checking, lexical upvalues for closures, and frame-pointer call unwinding.

Binary Bytecode Container (.ufc Format)

Precompiled Unfish bytecode modules are structured in a binary format with fixed endianness:

+-----------------------------------------------------------------------+
| Magic Bytes: 0x55 0x46 0x56 0x4D ("UFVM")               (4 bytes)     |
+-----------------------------------------------------------------------+
| Version: Major (u8), Minor (u8), Patch (u8), Flags (u8) (4 bytes)     |
+-----------------------------------------------------------------------+
| Constant Pool Count: uint32_t                           (4 bytes)     |
+-----------------------------------------------------------------------+
| Constant Pool Entries: [Type: u8, Length: u32, Data: bytes ...]       |
+-----------------------------------------------------------------------+
| Bytecode Instruction Stream: [Opcode: u8, Operands: var-length ...]   |
+-----------------------------------------------------------------------+
| Source Line Debug Mapping Table: [Offset: u32, Line: u32 ...]         |
+-----------------------------------------------------------------------+

Complete 59-Opcode Instruction Table

🚀 256-Register VM Architecture (RegVM)

While the stack VM provides a compact bytecode footprint, real-world interpretation often incurs high memory traffic due to constant operand pushes and pops. The Unfish RegVM provides a high-throughput alternative based on a 3-address register architecture.

32-Bit Fixed Instruction Formats

All RegVM instructions are encoded as 32-bit words for optimal cache line alignment and CPU branch prediction:

Format ABC:  [ Opcode : 8b ] [ Reg A : 8b ] [ Reg B : 8b ] [ Reg C : 8b ]
Format ABx:  [ Opcode : 8b ] [ Reg A : 8b ] [ Constant/Offset Bx : 16b  ]
Format AsBx: [ Opcode : 8b ] [ Reg A : 8b ] [ Signed Offset sBx : 16b   ]

Stack VM vs. Register VM Trace Comparison

Consider the arithmetic statement: let result = (a * b) + (c - d)

OP_LOAD_LOCAL  0    # push a
OP_LOAD_LOCAL  1    # push b
OP_MUL              # pop b, pop a, push (a*b)
OP_LOAD_LOCAL  2    # push c
OP_LOAD_LOCAL  3    # push d
OP_SUB              # pop d, pop c, push (c-d)
OP_ADD              # pop (c-d), pop (a*b), push sum
OP_STORE_LOCAL 4    # store into result

ROP_MUL  r4, r0, r1   # r4 = r0 * r1
ROP_SUB  r5, r2, r3   # r5 = r2 - r3
ROP_ADD  r4, r4, r5   # r4 = r4 + r5 (result)

Register Windowing & Calling Convention

Each execution fiber maintains an activation register file of 256 registers. Upon invoking a function:

• The caller places function arguments into consecutive registers starting at r(A+1).
• The call instruction ROP_CALL A, B, C shifts the callee's frame base pointer to rA.
• Inside the callee, argument values map directly to r0, r1, ... without copying.
• Upon return, ROP_RETURN A transfers the return value back to destination register rC.

🔌 C99 AOT & Host Embedding Guide

Unfish was engineered from line 1 to be embeddable into host C and C++ applications. The entire runtime is self-contained in unfish_runtime.h with zero external dependencies beyond standard C99.

Compiling to Native Standalone C99

You can transpile any Unfish source code directly into portable ANSI C99 source code:

# Transpile Unfish to C99 source file
unfish emit-c -o game_logic.c game_logic.uf

# Compile directly to a native binary using GCC or Clang
gcc -O3 game_logic.c -lm -o game_logic

Embedding Unfish in a C Host Application

Here is a complete, working host program that embeds Unfish, executes scripts, and registers a custom C function callable from Unfish:

/* host_application.c — Embedding Unfish in C */
#include "unfish_runtime.h"

/* Custom native function exposed to Unfish */
static UfVal native_ocean_depth(void* env, size_t argc, UfVal* args) {
    (void)env;
    double trench_id = (argc > 0 && uf_is_num(args[0])) ? uf_to_num(args[0]) : 0.0;
    printf("[Host C] Fetching bathymetric depth for trench #%.0f\n", trench_id);
    return uf_num(10994.0); /* Mariana Trench depth in meters */
}

int main(void) {
    printf("=== Starting Host Application with Embedded Unfish ===\n");

    /* 1. Initialize host runtime context */
    UfRuntime* rt = uf_runtime_new();

    /* 2. Register native C function into Unfish globals */
    uf_register_native(rt, "get_ocean_depth", native_ocean_depth);

    /* 3. Execute Unfish script from string */
    const char* script =
        "let depth = get_ocean_depth(1)\n"
        "say f'Submersible reached depth: {depth} meters!'\n"
        "let pressure = (depth / 10.0) + 1.0\n"
        "say f'Ambient pressure: {round(pressure)} atmospheres'\n";

    UfValue result = uf_eval_string(rt, script);

    /* 4. Cleanup runtime memory */
    uf_runtime_free(rt);
    printf("=== Unfish runtime cleanly terminated ===\n");
    return 0;
}

Bare-Metal & Microcontroller Deployment (-DUF_EMBEDDED)

For microcontrollers (ARM Cortex-M, ESP32, RISC-V) without standard POSIX memory allocators, compile with -DUF_EMBEDDED. Unfish replaces dynamic malloc with a fixed static buffer and routes text output to a custom UART transmitter:

/* Direct serial output to UART hardware */
void uart_send_char(char c) {
    while (!(USART1->SR & USART_SR_TXE));
    USART1->DR = c;
}

int main(void) {
    /* Route Unfish say statements to hardware UART */
    uf_set_putchar(uart_send_char);

    /* Runs fully within 256 KB static RAM buffer */
    uf_run_compiled_embedded();
    while (1);
}

⚖️ Language Comparison Matrix

Unfish combines the syntax elegance of Python, the lightweight embeddability of Lua, and the concurrency ergonomics of Go into a unified, zero-dependency C99 engine.