Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 22 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [1.11.0] - 2025-12-03

### Added

- **File Exclusion Feature**: New `--exclude` flag for filtering files using glob patterns
- Support for simple wildcards (`*.map`, `*.txt`) and directory patterns (`test/**/*.js`)
- Multiple pattern formats: repeated flags (`--exclude *.map --exclude *.md`) and comma-separated (`--exclude="*.map,*.md"`)
- Default exclusions for common system and development files (`.DS_Store`, `Thumbs.db`, `.git`, `.svn`, `*.swp`, `*~`, `.gitignore`, `.gitattributes`)
- Cyan-colored console output showing excluded file count and list
- Cross-platform path normalization for Windows compatibility
- Uses `picomatch` library (transitive dependency via `tinyglobby`) for pattern matching
- Unit testing infrastructure using Vitest
- Comprehensive test coverage (~68%) for core modules:
- `commandLine.ts` (84.56%): CLI argument parsing and validation
- `file.ts` (100%): File operations and duplicate detection
- Comprehensive test coverage (~72%) for core modules:
- `commandLine.ts` (~90%): CLI argument parsing and validation including exclude patterns
- `file.ts` (100%): File operations, duplicate detection, and exclusion filtering
- `cppCode.ts` (96.62%): C++ code generation and templates
- `consoleColor.ts` (100%): Console utilities
- `consoleColor.ts` (100%): Console utilities including new `cyanLog` function
- 15 new unit tests for file exclusion feature (8 for CLI parsing, 7 for exclusion logic)
- Test fixtures for validating file processing
- Coverage reports with HTML output
- Development documentation for testing in README.md and CLAUDE.md
- `cyanLog()` color function for exclusion output messages

### Changed

- Updated file collection pipeline to filter excluded files after pre-compressed detection
- Enhanced `commandLine.ts` with exclude pattern parsing for three argument formats
- Enhanced `file.ts` with `isExcluded()` function and exclusion reporting
- Updated `.gitignore` to exclude `coverage/` directory
- Enhanced documentation with testing sections
- Enhanced documentation with testing sections and comprehensive file exclusion guide
- Updated README.md with file exclusion section including usage examples, pattern syntax, and default exclusions
- Updated CLAUDE.md with technical file exclusion implementation details and example output

## [1.10.0] - 2025-11-20

Expand Down Expand Up @@ -251,6 +266,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- CLI interface with `-s`, `-e`, `-o` options
- `index.html` automatic default route handling

[1.11.0]: https://github.com/BCsabaEngine/svelteesp32/compare/v1.10.0...v1.11.0
[1.10.0]: https://github.com/BCsabaEngine/svelteesp32/compare/v1.9.4...v1.10.0
[1.9.4]: https://github.com/BCsabaEngine/svelteesp32/compare/v1.9.3...v1.9.4
[1.9.3]: https://github.com/BCsabaEngine/svelteesp32/compare/v1.9.2...v1.9.3
Expand Down
52 changes: 51 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@ npx svelteesp32 -e espidf -s ./dist -o ./output.h --etag=true --gzip=true

# Test the CLI directly using tsx (for development)
npx tsx src/index.ts -e psychic -s ./demo/svelte/dist -o ./output.h --etag=true --gzip=true

# Generate header with file exclusions
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="*.map" --exclude="*.md"
```

## Architecture
Expand All @@ -90,7 +93,7 @@ npx tsx src/index.ts -e psychic -s ./demo/svelte/dist -o ./output.h --etag=true

### Processing Pipeline

1. **File Collection** (`src/file.ts`): Scans source directory recursively using glob, skips pre-compressed files (.gz, .br) if originals exist, detects duplicate files via SHA256 hashing
1. **File Collection** (`src/file.ts`): Scans source directory recursively using glob, skips pre-compressed files (.gz, .br) if originals exist, filters files matching exclude patterns, detects duplicate files via SHA256 hashing
2. **Content Analysis** (`src/index.ts`): Determines MIME types using `mime-types` library, calculates MD5 hashes for ETag generation, groups files by extension for statistics
3. **Compression** (`src/index.ts`): Applies gzip level 9 compression, uses compressed version only when size reduction >15% and original >1024 bytes
4. **Code Generation** (`src/cppCode.ts`, `src/cppCodeEspIdf.ts`): Uses Handlebars templates with custom helpers (switch/case), generates optimized engine-specific C++ code with:
Expand All @@ -112,11 +115,58 @@ npx tsx src/index.ts -e psychic -s ./demo/svelte/dist -o ./output.h --etag=true
- **Automatic Gzip Compression**: Compresses assets when size reduction >15% and >1024 bytes
- **ETag Support**: HTTP cache validation for reduced network traffic with 304 Not Modified responses across all engines (psychic, psychic2, async, espidf)
- **Cache Control**: Configurable browser caching with `--cachetime`
- **File Exclusion**: Exclude unwanted files using glob patterns with `--exclude` option
- **Multi-Engine Support**: Generate code for different ESP web server libraries
- **File Type Analysis**: Groups files by extension with count statistics
- **Memory Optimization**: Binary data stored as const arrays in program memory
- **Optimized C++ Code**: Generated code uses modern C++ best practices with minimal overhead

### File Exclusion

The tool supports flexible file exclusion using glob patterns to filter out unwanted files from being embedded in the ESP32 firmware.

**Pattern Support:**

- Simple wildcards: `*.map`, `*.txt`
- Directory patterns: `test/**/*.js`, `docs/**/*`
- Specific files: `.DS_Store`, `README.md`

**Multiple Patterns:**

- Repeated flag: `--exclude *.map --exclude *.md`
- Comma-separated: `--exclude="*.map,*.md,*.txt"`
- Combined: Both formats can be mixed

**Default Exclusions:**
System and development files are excluded by default:

- `.DS_Store`, `Thumbs.db` (system files)
- `.git`, `.svn` (version control)
- `*.swp`, `*~` (editor files)
- `.gitignore`, `.gitattributes` (git config)

**Implementation:**

- Uses `picomatch` library (transitive dependency via `tinyglobby`)
- Filtering occurs after pre-compressed file detection but before file content reading
- Excluded files are logged with count and list for verification
- Windows path normalization for cross-platform consistency

**Example Output:**

```
Collecting source files

Excluded 3 file(s):
- assets/index.js.map
- README.md
- test/unit.test.js

Translation to header file
[index.html] ✓ gzip used (472 -> 308 = 65%)
...
```

## Development Environment

### Build System
Expand Down
66 changes: 66 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,71 @@ At the same time, it can be an advantage that the content is cached by the brows

Typically, the entry point for web applications is the **index.htm or index.html** file. This does not need to be listed in the browser's address bar because web servers know that this file should be served by default. Svelteesp32 also does this: if there is an index.htm or index.html file, it sets it as the main file to be served. So using `http://esp_xxx.local` or just entering the `http://x.y.w.z/` IP address will serve this main file.

### File Exclusion

The `--exclude` option allows you to exclude files from being embedded in the ESP32 firmware using glob patterns. This is useful for excluding source maps, documentation, and test files that shouldn't be part of the deployed application.

#### Basic Usage

```bash
# Exclude source maps
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="*.map"

# Exclude documentation files
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="*.md"

# Exclude multiple file types (comma-separated)
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="*.map,*.md,*.txt"

# Exclude using multiple flags
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="*.map" --exclude="*.md"

# Exclude entire directories
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="test/**/*"

# Combine multiple approaches
npx svelteesp32 -e psychic -s ./dist -o ./output.h --exclude="*.map,*.md" --exclude="docs/**/*"
```

#### Pattern Syntax

The exclude patterns use standard glob syntax:

- `*.map` - Match all files ending with `.map`
- `**/*.test.js` - Match all `.test.js` files in any directory
- `test/**/*` - Match all files in the `test` directory and subdirectories
- `.DS_Store` - Match specific filename

#### Default Exclusions

By default, the following system and development files are automatically excluded:

- `.DS_Store` (macOS system file)
- `Thumbs.db` (Windows thumbnail cache)
- `.git` (Git directory)
- `.svn` (SVN directory)
- `*.swp` (Vim swap files)
- `*~` (Backup files)
- `.gitignore` (Git ignore file)
- `.gitattributes` (Git attributes file)

Custom exclude patterns are added to these defaults.

#### Exclusion Output

When files are excluded, you'll see a summary in the build output:

```
Excluded 5 file(s):
- assets/index.js.map
- assets/vendor.js.map
- README.md
- docs/guide.md
- test/unit.test.js
```

This helps you verify that the correct files are being excluded from your build.

### C++ defines

To make it easy to integrate into a larger c++ project, we have made a couple of variables available as c++ defines.
Expand Down Expand Up @@ -352,6 +417,7 @@ You can use the following c++ directives at the project level if you want to con
| `-s` | **Source dist folder contains compiled web files** | (required) |
| `-e` | The engine for which the include file is created (psychic/psychic2/async/espidf) | psychic |
| `-o` | Generated output file with path | `svelteesp32.h` |
| `--exclude` | Exclude files matching glob pattern (repeatable or comma-separated) | Default system files |
| `--etag` | Use ETag header for cache (true/false/compiler) | false |
| `--cachetime` | Override no-cache response with a max-age=\<cachetime\> response (seconds) | 0 |
| `--gzip` | Compress content with gzip (true/false/compiler) | true |
Expand Down
Loading