Tauq
Token-efficient data notation for the AI era. 44% fewer tokens than JSON, verified with tiktoken. Schema-driven, streaming-capable, and programmable.
Verified Token Savings
All counts verified with tiktoken cl100k_base (GPT-4/Claude tokenizer).
1000 Records Benchmark
JSON (24 tokens)
[{"id": 1, "name": "Alice"},
{"id": 2, "name": "Bob"}]Tauq (11 tokens)
!def User id name 1 Alice 2 Bob
Why Tauq?
Token-Optimal
44-54% fewer tokens than JSON. Space delimiters tokenize better than commas.
True Streaming
StreamingParser iterator API. Process records one at a time, no count required.
Schema-Driven
Define data shapes with !def, switch schemas with !use. Nested types and typed arrays.
Programmable
Tauq Query for data transformations with Unix pipe model and polyglot support.
Production CLI
Build, format, minify, exec, and validate commands out of the box.
Multi-Language
Rust, Python, JavaScript, Go, Java, C#, Swift bindings available.
Syntax Examples
Simple Values
name "Alice" age 30 active true role admin # Barewords ok
Tabular Data
!def User id name email role 1 Alice "alice@ex.com" admin 2 Bob "bob@ex.com" user 3 Carol "carol@ex.com" user
Nested Types
!def Address street city
!def User id name addr:Address
1 Alice { "123 Main" "NYC" }
2 Bob { "456 Oak" "LA" }Minified
!def U id name; 1 Alice; 2 Bob
All on one line for maximum compression!
CLI Commands
tauq build data.tqnParse Tauq to JSON
tauq format data.jsonConvert JSON to Tauq
tauq minify data.tqnCompress to single line
tauq exec pipeline.tqqRun Tauq Query
tauq validate data.tqnCheck syntax
cat data.tqn | tauq build -Stdin support
Installation
Rust
cargo install tauq
Python
pip install tauq
JavaScript
npm install tauq
Go
go get github.com/epistates/tauq