Tasmota/lib/libesp32/berry/REPOSITORY_MAP.md

# Berry Repository Structure Map

## Overview
Berry is an ultra-lightweight dynamically typed embedded scripting language designed for lower-performance embedded devices. The interpreter core is less than 40KiB and can run on less than 4KiB heap.

## Directory Structure

### `/src/` - Core Source Code (152 files)
**Main Components:**
- **Virtual Machine**: `be_vm.c` (1419 lines) - Register-based VM execution
- **Parser**: `be_parser.c` (1841 lines) - One-pass compiler and syntax analysis  
- **Lexer**: `be_lexer.c` (914 lines) - Tokenization and lexical analysis
- **API**: `be_api.c` (1179 lines) - External C API interface
- **Code Generation**: `be_code.c` (983 lines) - Bytecode generation
- **Garbage Collector**: `be_gc.c` (613 lines) - Mark-sweep garbage collection

**Data Types & Objects:**
- **Strings**: `be_string.c` (326 lines), `be_strlib.c` (1137 lines)
- **Lists**: `be_list.c` (207 lines), `be_listlib.c` (556 lines)  
- **Maps**: `be_map.c` (354 lines), `be_maplib.c` (265 lines)
- **Classes**: `be_class.c` (374 lines)
- **Functions**: `be_func.c` (183 lines)
- **Bytes**: `be_byteslib.c` (1992 lines) - Binary data handling

**Built-in Libraries:**
- **JSON**: `be_jsonlib.c` (645 lines) - JSON parsing/generation
- **Math**: `be_mathlib.c` (438 lines) - Mathematical functions
- **OS**: `be_oslib.c` (271 lines) - Operating system interface
- **File**: `be_filelib.c` (265 lines) - File I/O operations
- **Debug**: `be_debug.c` (418 lines), `be_debuglib.c` (289 lines)
- **Introspection**: `be_introspectlib.c` (298 lines)
- **Time**: `be_timelib.c` (72 lines)

**Memory & Execution:**
- **Memory Management**: `be_mem.c` (377 lines)
- **Execution**: `be_exec.c` (531 lines)
- **Bytecode**: `be_bytecode.c` (634 lines)
- **Variables**: `be_var.c` (201 lines)
- **Modules**: `be_module.c` (509 lines)

**Headers:**
- **Main Header**: `berry.h` (2395 lines) - Primary API definitions
- **Constants**: `be_constobj.h` (505 lines) - Constant object definitions

### `/tests/` - Unit Tests (54 files)
**Core Language Tests:**
- `assignment.be`, `bool.be`, `class.be`, `closure.be`, `function.be`
- `for.be`, `vararg.be`, `cond_expr.be`, `exceptions.be`

**Data Type Tests:**
- `list.be`, `map.be`, `range.be`, `string.be`, `int.be`, `bytes.be`

**Library Tests:**
- `json.be` (9168 lines) - **Comprehensive JSON security tests**
- `math.be`, `os.be`, `debug.be`, `introspect.be`

**Parser & Compiler Tests:**
- `parser.be`, `lexer.be`, `compiler.be`, `suffix.be`

**Advanced Feature Tests:**
- `virtual_methods.be`, `super_auto.be`, `class_static.be`
- `division_by_zero.be`, `reference.be`, `compound.be`

### `/examples/` - Example Programs (16 files)
- `fib_rec.be` - Fibonacci recursion
- `qsort.be` - Quick sort implementation
- `bintree.be` - Binary tree operations
- `json.be` - JSON usage examples
- `repl.be` - REPL implementation

### `/default/` - Default Configuration (17 files)
- `berry_conf.h` - Configuration settings
- `be_modtab.c` - Module table definitions
- `be_port.c` - Platform-specific code
- `berry.c` - Main executable entry point

### `/generate/` - Generated Files (31 files)
- `be_const_strtab.h` - String table constants
- `be_fixed_*.h` - Fixed/compiled module definitions
- Auto-generated constant definitions

### `/tools/` - Development Tools
**Code Generation:**
- `/coc/` - Compile-on-command tools (13 files)
- Python scripts for code generation and optimization

**Editor Support:**
- `/plugins/vscode/` - Visual Studio Code plugin
- `/plugins/Notepad++/` - Notepad++ syntax highlighting

**Grammar:**
- `berry.ebnf` - EBNF grammar definition
- `berry.bytecode` - Bytecode format specification

## Key Architecture Components

### 1. **Virtual Machine** (`be_vm.c`)
- Register-based VM (not stack-based)
- Optimized for low memory usage
- Handles instruction execution and control flow

### 2. **Parser & Lexer** (`be_parser.c`, `be_lexer.c`)
- One-pass compilation
- Generates bytecode directly
- Error handling and recovery

### 3. **Memory Management** (`be_mem.c`, `be_gc.c`)
- Custom memory allocator
- Mark-sweep garbage collector
- Low memory footprint optimization

### 4. **Type System** 
- **Value Types**: int, real, boolean, string (not class objects)
- **Object Types**: list, map, range, class instances
- Optimized for performance vs. pure OOP

### 5. **Security Features** (Recently Added)
- **JSON Security**: Comprehensive buffer overflow protection
- Unicode handling with proper size calculation
- Input validation and sanitization

## Recent Security Work

### JSON Parser Security (`be_jsonlib.c`)
- **Fixed**: Critical buffer overflow in Unicode handling
- **Added**: Comprehensive security tests (10 test functions)
- **Implemented**: Safe string length calculation
- **Protected**: Against memory exhaustion attacks

## Build System
- **Makefile** - Primary build system
- **CMakeLists.txt** - CMake support
- **library.json** - PlatformIO library definition

## Testing Infrastructure
- **51 unit tests** covering all major features
- **Automated test runner** via `make test`
- **Security regression tests** for vulnerability prevention
- **Cross-platform compatibility tests**

## File Statistics
- **Total Source Files**: ~200 files
- **Core C Code**: ~24,000 lines
- **Test Code**: ~15,000 lines  
- **Documentation**: Comprehensive README and examples
- **Binary Size**: <40KiB interpreter core
- **Memory Usage**: <4KiB heap minimum

This repository represents a complete, production-ready embedded scripting language with comprehensive testing, security features, and development tools.