Compare commits

..

47 Commits

Author SHA1 Message Date
WerWolv
4299243e95 includes/std: Added hex::dec::lz4_decompress() 2024-07-07 15:51:40 +02:00
Surasia
23b3c2b5d0 git: Fix hinf_tag and hinf_module links in readme (#268) 2024-07-07 13:44:04 +02:00
Surasia
da1c2687e8 patterns/hinf_*: Add Halo Infinite tag and module file patterns (#267)
* patterns/hinf: Add Halo Infinite Tag and Module patterns

* patterns/hinf_bitmap: remove pattern
2024-07-06 17:33:03 +02:00
WerWolv
18f968df5b includes/std: Drastically improved std::mem::MagicSearch, added find_string and find_string_in_range 2024-07-04 20:47:39 +02:00
WerWolv
f797c711ca includes/std: Optimize std::bit::popcount 2024-07-04 17:58:12 +02:00
WerWolv
5380877da4 tests: Added error message when no description is used 2024-07-04 08:28:20 +02:00
Marc Butler
bb46276bde patterns: Added Password Safe Version 3 pattern (#266)
* Add password safe V3

* Added description pragma

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2024-07-04 08:27:51 +02:00
Nik
4242869ac1 patterns/zlib: Cast bitfield enums to integers before doing math with them 2024-07-03 22:48:15 +02:00
WerWolv
8bdcd814a9 includes/std: Added std::unimplemented() 2024-07-02 18:00:52 +02:00
WerWolv
d7811ff5e9 patterns: Replaced old bitfield order attributes with new ones
Fixes #265
2024-07-02 18:00:41 +02:00
WerWolv
4d0b3e21bc includes/std: Added better documentation to std::mem::current_bit_offset() 2024-07-02 18:00:05 +02:00
WerWolv
acd6903b21 includes/std: Added crc8, crc16 and crc64 functions
Fixes #226
2024-07-02 17:59:40 +02:00
WerWolv
8d2a39f7d4 git: Added build folders to gitignore 2024-07-02 17:58:47 +02:00
Nik
65ef3a7783 includes/std: Removed cyclic dependency between std.mem and std.string 2024-07-01 23:47:09 +02:00
Nik
21a0b99eac includes/std: Added missing string include to std::mem 2024-06-29 23:25:58 +02:00
Rebuild
a33b68921d patterns: Added ttf/otf format (#263)
* Add ttf/otf format

* ttf: Put cff to preprocessing
2024-06-29 22:52:21 +02:00
Nik
a0bb71be58 includes/std: Added std::core::execute_function() 2024-06-29 11:53:43 +02:00
WerWolv
c7fc39ac19 git: Make sure test data is always treated as binaries 2024-06-26 20:59:41 +02:00
WerWolv
b540ead0ae includes/std: Added std::IIndexed as a nicer wrapper around std::core::array_index() 2024-06-25 22:52:57 +02:00
Nik
26878548f0 includes/std: Fixed NullString not including null terminator and not checking for string end correctly 2024-06-23 18:45:13 +02:00
C3pa
e4c9d86755 includes/std: Add some fixes from the WerWolv/Documentation repo (#262)
* Escape backslash in DOSPath docs

* Spelling correction for std\sys.pat

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2024-06-22 10:59:36 +02:00
WerWolv
2a3de1b705 includes: Tabs -> Spaces 2024-06-22 10:46:30 +02:00
WerWolv
e02280f9ee includes/type: Added arbitrarily formattable type 2024-06-20 21:21:42 +02:00
WerWolv
bbba68cef7 includes/std: Added null-terminated strings to the strings library 2024-06-20 21:21:30 +02:00
Joachim Schiele
28b281b403 patterns/pe: Added .didata section support (#257) 2024-06-16 15:23:01 +02:00
Kirill Gladkikh
c807959d75 patterns/jpeg: Support EOF in the middle of JPEG (#240)
* Support EOF in the middle of file

* Changed to eoi marker searching
2024-06-16 15:16:01 +02:00
Francisco J. Solis
3416d30f2b pattern/dds: Fix bitfield PixelFormatFlags (#259)
* Fix bitfield PixelFormatFlags

The padding names should not repeat
Comments should be added to each field for clarification
Padding should be corrected for luminance.

* Re-work of PixelFormatFlags

Add more information considering current GIMP repository structure
https://gitlab.gnome.org/GNOME/gimp/-/blame/master/plug-ins/file-dds/dds.h\#L145
2024-06-09 22:15:05 +02:00
iTrooz
10bf1c76cf chore: mention description pragma in PR template 2024-05-20 01:05:48 +02:00
iTrooz
5b7c212029 git: use ccache (#253) 2024-05-20 01:01:59 +02:00
iTrooz
1c2e948940 test: Make description pragma mandatory (#251) 2024-05-20 00:35:24 +02:00
iTrooz
7ea34e410a fix: add missing ; to core.pat 2024-05-20 00:13:52 +02:00
iTrooz
56950e44d8 fix: use renamed PatternRuntime::getEvalError() (#252) 2024-05-20 00:10:34 +02:00
iTrooz
ed42452fc1 chore: remove old coverage code 2024-05-20 00:01:36 +02:00
iTrooz
a68ecb8888 fix: run tests concurrently 2024-05-20 00:01:00 +02:00
WerWolv
8d4cf59497 patterns/ffx: Updated submodule 2024-05-19 17:05:56 +02:00
Benjamin Gilbert
3c2fed22e4 patterns: Add Windows animated cursor (#242)
* patterns: Add Windows animated cursor

* Added readme entry

* Added mime type to readme

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2024-05-19 16:58:03 +02:00
Christoph Koehler
ff550bd105 patterns: Add Digital Terrain Elevation Data (DTED) pattern file (#243)
Tested on DTED0 and DTED2, will likely work with DTED1 as well.
2024-05-19 16:55:30 +02:00
Rebuild
0ad6e3abde patterns/mp4: Use FixedPoint32 instead of u32 for width and height (#248)
Update mp4.hexpat

* Use FixedPoint32 instead of u32 for width and height
* Add formatting for displaying fixed point 16.16 and 8.8 numbers
2024-05-19 16:55:04 +02:00
Marijn Kneppers
891968db53 includes/std: Add create_directories to std (#250) 2024-05-19 16:46:05 +02:00
Marijn Kneppers
5799d1d7ea includes/std: Make std::file::flush not delete files (#246) 2024-05-12 23:39:49 +02:00
Marijn Kneppers
9c6bfeb50f includes/hex: Added hex::core::add_virtual_file (#247) 2024-05-12 23:38:57 +02:00
AdventureT
9f7cdded6a Crashlvl: Added support for version 0.95 (#245)
* Working on new Version

* Needs testing

* Updated CrashLvl script to support version 0.94c

* Update README.md

* Added TestData and author to script

* Delete tests/patterns/test_data/CrashLvl.hexpat.bin

* CrashLvl.hexpat: Added support for version 0.95
2024-05-12 23:38:25 +02:00
Claudius Coenen
03298b0b0c patterns: Added Wintec TES file pattern (#238) 2024-04-21 12:33:19 +02:00
LolHacksRule
5f2738872e patterns: Added XGSPak and XGSTexture patterns (#237)
* Add XGSPak and XGSTexture patterns

* Whoops forgot proper pointers

* Fix typo

* Whoops

* Proper start

* Update xgstexture.hexpat
2024-04-21 12:32:26 +02:00
nazDridoy
63585e6de7 patterns/7z: Added MIME type (#241) 2024-04-21 12:25:00 +02:00
Sabhya Raj Mehta
cb6caafa64 patterns/refs: Added Structure for MinStoreB+ node, missing GlobalRootNode Names & parses all Superblocks (#236)
[+] Added Structure for MinStoreB+ node, Missing GlobalRootNode Names
2024-03-29 11:55:01 +01:00
Leon Jacobs
236fadee47 patterns: Added gguf pattern (#235)
* (feat) add gguf parser

* (chore) cleanup empty struct

* (tests) add sample gguf for tests and update readme

* (feat) update type enum
2024-03-28 21:55:18 +01:00
65 changed files with 3610 additions and 1871 deletions

2
.gitattributes vendored
View File

@@ -1,4 +1,4 @@
*.pat linguist-language=Rust
*.hexpat linguist-language=Rust
tests/patterns/test_data/* binary
tests/patterns/test_data/** binary

View File

@@ -5,6 +5,7 @@
## Checklist
- [ ] A pattern for this format doesn't exist yet (or this PR improves the existing one)
- [ ] The new pattern has been added to the relevant table in the Readme
- [ ] The new pattern has a description pragma (`#pragma description My pattern Description here`)
- [ ] The pattern was associated with all relevant MIME types (using `#pragma MIME mime-type` in the source code)
- Make sure to never use `application/octet-stream` here as that means "Unidentifiable binary data"
- [ ] A test file for this pattern has been added to [/tests/patterns/test_data](/tests/patterns/test_data)

View File

@@ -28,6 +28,7 @@ jobs:
sudo apt update
sudo apt install -y \
build-essential \
ccache \
gcc-12 \
g++-12 \
lld \
@@ -41,12 +42,21 @@ jobs:
sudo pip install jsonschema
- name: 📜 Setup ccache
uses: hendrikmuhs/ccache-action@v1
with:
key: ${{ runner.os }}-ccache-${{ github.run_id }}
restore-keys: ${{ runner.os }}-ccache
max-size: 1G
- name: 🛠️ Build
run: |
cd tests
mkdir -p build
cd build
CC=gcc-12 CXX=g++-12 cmake \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_FLAGS="-fuse-ld=lld --coverage" \
-DCMAKE_CXX_FLAGS="-fuse-ld=lld --coverage" \
-DLIBPL_ENABLE_TESTS=OFF \
@@ -57,20 +67,7 @@ jobs:
- name: 🧪 Perform Unit Tests
run: |
cd tests/build
ctest --output-on-failure
#- name: ⚗️ Generate Coverage Report
# run: |
# cd tests/build
# lcov -d ./_deps/pattern_language-build/lib --gcov-tool /usr/bin/gcov-12 -c -o coverage.info
#- name: ⬆️ Upload Coverage Report
# uses: coverallsapp/github-action@master
# env:
# NODE_OPTIONS: --max-old-space-size=8000
# with:
# github-token: ${{ secrets.GITHUB_TOKEN }}
# path-to-lcov: tests/build/coverage.info
ctest --output-on-failure -j 4
- name: 📎 Validate JSON Files
run: |

3
.gitignore vendored
View File

@@ -1,5 +1,6 @@
tests/cmake-build-debug/
tests/cmake*/
tests/build*/
.idea/
.DS_Store

View File

@@ -1,6 +1,7 @@
# ImHex Database
This repository serves as a database for files to use with the [ImHex Hex Editor](https://github.com/WerWolv/ImHex). It currently contains
- [Patterns](/patterns) - Binary Format definitions for the Pattern Language
- [Pattern Libraries](/includes) - Libraries that make using the Pattern Language easier
- [Magic Files](/magic) - Custom magic file definitions for the use with libmagic
@@ -25,6 +26,7 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| 3DS | | [`patterns/3ds.hexpat`](patterns/3ds.hexpat) | Autodesk 3DS Max Model file |
| 7Z | | [`patterns/7z.hexpat`](patterns/7z.hexpat) | 7z File Format |
| AFE2 | | [`patterns/afe2.hexpat`](patterns/afe2.hexpat) | Nintendo Switch Atmosphère CFW Fatal Error log |
| ANI | `application/x-navi-animation` | [`patterns/ani.hexpat`](patterns/ani.hexpat) | Windows Animated Cursor file |
| AR | `application/x-archive` | [`patterns/ar.hexpat`](patterns/ar.hexpat) | Static library archive files |
| ARIA2 | | [`patterns/aria2.hexpat`](patterns/aria2.hexpat) | ARIA2 Download Manager Control files |
| ARM VTOR | | [`patterns/arm_cm_vtor.hexpat`](patterns/arm_cm_vtor.hexpat) | ARM Cortex M Vector Table Layout |
@@ -48,6 +50,7 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| DMG | | [`patterns/dmg.hexpat`](patterns/dmg.hexpat) | Apple Disk Image Trailer (DMG) |
| DS_Store | | [`patterns/dsstore.hexpat`](patterns/dsstore.hexpat) | .DS_Store file format |
| DTA | | [`patterns/max_v104.hexpat`](patterns/max_v104.hexpat) | Mechanized Assault and Exploration v1.04 (strategy game) save file format |
| DTED | | [`patterns/dted.hexpat`](patterns/dted.hexpat) | Digital Terrain Elevation Data (DTED) |
| ELF | `application/x-executable` | [`patterns/elf.hexpat`](patterns/elf.hexpat) | ELF header in elf binaries |
| EVTX | | [`patterns/evtx.hexpat`](patterns/evtx.hexpat) | MS Windows Vista Event Log |
| FAS | | [`patterns/fas_oskasoftware.hexpat`](patterns/fas_oskasoftware.hexpat) [`patterns/fas_oskasoftware_old.hexpat`](patterns/fas_oskasoftware_old.hexpat) (Old versions of Oska DeskMate) | Oska Software DeskMates FAS (Frames and Sequences) file |
@@ -56,9 +59,11 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| File System | | [`patterns/fs.hexpat`](patterns/fs.hexpat) | Drive File System |
| FLAC | `audio/flac` | [`patterns/flac.hexpat`](patterns/flac.hexpat) | Free Lossless Audio Codec, FLAC Audio Format |
| GB | `application/x-gameboy-rom` | [`patterns/gb.hexpat`](patterns/gb.hexpat) | Gameboy ROM |
| GGUF | | [`patterns/gguf.hexpat`](patterns/gguf.hexpat) | GGML Inference Models |
| GIF | `image/gif` | [`patterns/gif.hexpat`](patterns/gif.hexpat) | GIF image files |
| GZIP | `application/gzip` | [`patterns/gzip.hexpat`](patterns/gzip.hexpat) | GZip compressed data format |
| Halo Bitmap || [`patterns/hinf_bitmap.hexpat`](patterns/hinf_bitmap.hexpat) | Halo Infinite Bitmap tag files |
| Halo Tag || [`patterns/hinf_tag.hexpat`](patterns/hinf_tag.hexpat) | Halo Infinite Tag Files |
| Halo Module || [`patterns/hinf_module.hexpat`](patterns/hinf_module.hexpat) | Halo Infinite Module Archive Files |
| Halo HavokScript || [`patterns/hinf_luas.hexpat`](patterns/hinf_luas.hexpat) | Halo Infinite HavokScript 5.1 Bytecode |
| ICO | | [`patterns/ico.hexpat`](patterns/ico.hexpat) | Icon (.ico) or Cursor (.cur) files |
| ID3 | `audio/mpeg` | [`patterns/id3.hexpat`](patterns/id3.hexpat) | ID3 tags in MP3 files |
@@ -82,16 +87,18 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| NRO | | [`patterns/nro.hexpat`](patterns/nro.hexpat) | Nintendo Switch NRO files |
| NTAG | | [`patterns/ntag.hexpat`](patterns/ntag.hexpat) | NTAG213/NTAG215/NTAG216, NFC Forum Type 2 Tag compliant IC |
| OGG | `audio/ogg` | [`patterns/ogg.hexpat`](patterns/ogg.hexpat) | OGG Audio format |
| PAK | | [`patterns/xgspak.hexpat`](patterns/xgspak.hexpat) | Exient XGS Engine Pak files |
| PCAP | `application/vnd.tcpdump.pcap` | [`patterns/pcap.hexpat`](patterns/pcap.hexpat) | pcap header and packets |
| PCX | `application/x-pcx` | [`patterns/pcx.hexpat`](patterns/pcx.hexpat) | PCX Image format |
| PE | `application/x-dosexec` `application/x-msdownload` | [`patterns/pe.hexpat`](patterns/pe.hexpat) | PE header, COFF header, Standard COFF fields and Windows Specific fields |
| PP | | [`patterns/selinuxpp.hexpat`](patterns/selinuxpp.pat) | SE Linux package |
| PFS0 | | [`patterns/pfs0.hexpat`](patterns/pfs0.hexpat) | Nintendo Switch PFS0 archive (NSP files) |
| PIF | `image/pif` | [`patterns/pif.hexpat`](patterns/pif.hexpat) | PIF Image Format |
| PNG | `image/png` | [`patterns/png.hexpat`](patterns/png.hexpat) | PNG image files |
| PRODINFO | | [`patterns/prodinfo.hexpat`](patterns/prodinfo.hexpat) | Nintendo Switch PRODINFO |
| Protobuf | | [`patterns/protobuf.hexpat`](patterns/protobuf.hexpat) | Google Protobuf encoding |
| PyInstaller | | [`patterns/pyinstaller.hexpat`](patterns/pyinstaller.hexpat) | PyInstaller binray files |
| PNG | `image/png` | [`patterns/png.hexpat`](patterns/png.hexpat) | PNG image files |
| PRODINFO | | [`patterns/prodinfo.hexpat`](patterns/prodinfo.hexpat) | Nintendo Switch PRODINFO |
| Protobuf | | [`patterns/protobuf.hexpat`](patterns/protobuf.hexpat) | Google Protobuf encoding |
| psafe3 | | [`patterns/psafe3.hexpat`](patterns/psafe3.hexpat`) | Password Safe V3 |
| PyInstaller | | [`patterns/pyinstaller.hexpat`](patterns/pyinstaller.hexpat) | PyInstaller binray files |
| PYC | | [`patterns/pyc.hexpat`](patterns/pyc.hexpat) | Python bytecode files |
| QBCL | | [`patterns/qbcl.hexpat`](patterns/qbcl.hexpat) | Qubicle voxel scene project file |
| QOI | `image/qoi` | [`patterns/qoi.hexpat`](patterns/qoi.hexpat) | QOI image files |
@@ -107,6 +114,7 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| TAR | `application/x-tar` | [`patterns/tar.hexpat`](patterns/tar.hexpat) | Tar file format |
| TIFF | `image/tiff` | [`patterns/tiff.hexpat`](patterns/tiff.hexpat) | Tag Image File Format |
| TGA | `image/tga` | [`patterns/tga.hexpat`](patterns/tga.hexpat) | Truevision TGA/TARGA image |
| TTF | `font/ttf`, `font/otf` | [`patterns/ttf.hexpat`](patterns/ttf.hexpat) | TrueType and OpenType font format |
| Ubiquiti | | [`patterns/ubiquiti.hexpat`](patterns/ubiquiti.hexpat) | Ubiquiti Firmware (update) image |
| UEFI | | [`patterns/uefi.hexpat`](patterns/uefi.hexpat)` | UEFI structs for parsing efivars |
| UF2 | | [`patterns/uf2.hexpat`](patterns/uf2.hexpat) | [USB Flashing Format](https://github.com/microsoft/uf2) |
@@ -117,7 +125,8 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| WAS | | [`patterns\was_oskasoftware.hexpat`](patterns\was_oskasoftware.hexpat) | Oska Software DeskMates WAS/WA3 (WAVE/MP3 Set) file
| WAD | | [`patterns/wad.hexpat`](patterns/wad.hexpat) | DOOM WAD Archive |
| XBEH | `audio/x-xbox-executable` | [`patterns/xbeh.hexpat`](patterns/xbeh.hexpat) | Xbox executable |
| XCI | | [`patterns/xci.hexpat`](patterns/xci.hexpat) | Nintendo Switch XCI cardridge ROM |
| XCI | | [`patterns/xci.hexpat`](patterns/xci.hexpat) | Nintendo Switch XCI cartridge ROM |
| XGT | | [`patterns/xgt.hexpat`](patterns/xgstexture.hexpat) | Exient XGS Engine Texture |
| Xilinx BIT | | [`patterns/xilinx_bit.hexpat`](patterns/xilinx_bit.hexpat) | Xilinx FPGA Bitstreams |
| Xilinx Bootgen | | [`patterns/xilinx_bootgen.hexpat`](patterns/xilinx_bootgen.hexpat) | Xilinx ZynqMP Boot Images |
| ZIP | `application/zip` | [`patterns/zip.hexpat`](patterns/zip.hexpat) | End of Central Directory Header, Central Directory File Headers |
@@ -203,12 +212,14 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
> import custom encoding from File -> Import... -> Custome Encoding File
### Data Processor Nodes
| Name | Path | Description |
|------|------|-------------|
| Caesar Cipher | [`nodes/caesar.hexnode`](nodes/caesar.hexnode) | Simple adjustable per-byte Caecar Cipher (ROT) |
| XOR Cipher | [`nodes/xor.hexnode`](nodes/xor.hexnode) | XORs a input with a repeating XOR pad |
### Themes
| Name | Path | Description |
|------|------|-------------|
| Visual Studio Dark | [`themes/vs_dark.json`](themes/vs_dark.json) | Theme similar to Visual Studio's Dark theme |

View File

@@ -37,4 +37,14 @@ namespace auto hex::core {
return result;
};
/**
Add a file to the Virtual Filesystem
@param path The name of the file
@param pattern The pattern associated with the file
*/
fn add_virtual_file(str path, auto pattern)
{
builtin::hex::core::add_virtual_file(path, pattern);
};
}

View File

@@ -60,5 +60,16 @@ namespace auto hex::dec {
fn zstd_decompress(ref auto pattern, std::mem::Section section) {
return builtin::hex::dec::zstd_decompress(pattern, section);
};
/**
Decompresses the bytes of a pattern into a section using the lz4 algorithm
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@param frame Whether the data is framed or not
@return true if successful, false otherwise
*/
fn lz4_decompress(ref auto pattern, std::mem::Section section, bool frame = true) {
return builtin::hex::dec::lz4_decompress(pattern, section, frame);
};
}

View File

@@ -11,20 +11,20 @@ import hex.dec;
*/
namespace auto hex::type {
/**
A mangled name string that gets demangled when displayed
*/
struct MangledName {
char value[];
} [[sealed, format("hex::type::impl::format_mangled_name")]];
namespace impl {
fn format_mangled_name(ref MangledName name) {
return hex::dec::demangle(name.value);
};
}
}

View File

@@ -1,6 +1,7 @@
#pragma once
import std.sys;
import std.core;
/*!
The array library contains a helper type to make it easier to create multi-dimensional arrays
@@ -30,6 +31,16 @@ namespace auto std {
std::assert($ - startAddress == NumBytes, "Not enough bytes available to fit a whole number of types");
} [[format("std::impl::format_array")]];
/**
An interface type for getting the index of the currently processed element in an array. This is a nice wrapper around `std::core::array_index()`
To use it, inherit from it and use the `this.index` field to get the index of the current element
*/
struct IIndexed {
const u64 index = std::core::array_index();
};
namespace impl {
fn format_array(ref auto array) {

View File

@@ -3,61 +3,65 @@
import std.limits;
/*!
This library contains various helper functions for common bit operations.
This library contains various helper functions for common bit operations.
*/
namespace auto std::bit {
/**
Calculates the number of 1 bits in a given number
@param x The number
@return The number of bits set to 1 in `x`
*/
fn popcount(u128 x) {
x = (x & (std::limits::u128_max() / 3)) + ((x >> 1) & (std::limits::u128_max() / 3));
x = (x & (std::limits::u128_max() / 5)) + ((x >> 2) & (std::limits::u128_max() / 5));
x = (x & (std::limits::u128_max() / 17)) + ((x >> 4) & (std::limits::u128_max() / 17));
return x % 0xFF;
};
/**
Calculates the number of 1 bits in a given number
@param x The number
@return The number of bits set to 1 in `x`
*/
fn popcount(u128 x) {
const u128 a = 0x55555555555555555555555555555555;
const u128 b = 0x33333333333333333333333333333333;
const u128 c = 0x0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F;
/**
Checks if only a single bit is set to 1 in a given number
@param x The number
@return True if there's a single bit set to 1 in `x`, false otherwise
*/
fn has_single_bit(u128 x) {
return x != 0 && (x & (x - 1)) == 0;
};
/**
Rounds the given number up to the next bigger power of two
@param x The number
@return Next bigger power of two that can fit `x`
*/
fn bit_ceil(u128 x) {
if (x == 0) return 0;
u8 i;
while ((1 << i) < x)
i = i + 1;
return 1 << i;
};
x = (x & a) + ((x >> 1) & a);
x = (x & b) + ((x >> 2) & b);
x = (x & c) + ((x >> 4) & c);
return x % 0xFF;
};
/**
Rounds the given number down to the next bigger power of two
@param x The number
@return Next smaller power of two
*/
fn bit_floor(u128 x) {
if (x == 0) return 0;
u8 i;
while ((x >> i) > 0)
i = i + 1;
return 1 << (i - 1);
};
/**
Checks if only a single bit is set to 1 in a given number
@param x The number
@return True if there's a single bit set to 1 in `x`, false otherwise
*/
fn has_single_bit(u128 x) {
return x != 0 && (x & (x - 1)) == 0;
};
/**
Rounds the given number up to the next bigger power of two
@param x The number
@return Next bigger power of two that can fit `x`
*/
fn bit_ceil(u128 x) {
if (x == 0) return 0;
u8 i;
while ((1 << i) < x)
i = i + 1;
return 1 << i;
};
/**
Rounds the given number down to the next bigger power of two
@param x The number
@return Next smaller power of two
*/
fn bit_floor(u128 x) {
if (x == 0) return 0;
u8 i;
while ((x >> i) > 0)
i = i + 1;
return 1 << (i - 1);
};
}

View File

@@ -164,4 +164,13 @@ namespace auto std::core {
fn set_pattern_comment(ref auto pattern, str comment) {
builtin::std::core::set_pattern_comment(pattern, name);
};
/**
Executes the function with the given name, passing in all given arguments
@param function_name The namespace-prefixed name of the function
@param args Arguments to pass to the function
*/
fn execute_function(str function_name, auto ... args) {
builtin::std::core::execute_function(function_name, args);
};
}

View File

@@ -1,112 +1,120 @@
#pragma once
/*!
The File library allows reading and writing from/to external files using
a C-like File IO API.
The File library allows reading and writing from/to external files using
a C-like File IO API.
**These functions are considered dangerous and require the user to manually permit them**
**These functions are considered dangerous and require the user to manually permit them**
*/
namespace auto std::file {
/**
/**
A handle representing a file that has been opened
*/
using Handle = s32;
/**
using Handle = s32;
/**
The mode to open a file in.
Read opens the file in read-only mode
Write opens the file in read and write mode
Create creates a new file if it doesn't exist and overwrites an existing file
Read opens the file in read-only mode
Write opens the file in read and write mode
Create creates a new file if it doesn't exist and overwrites an existing file
*/
enum Mode : u8 {
Read = 1,
Write = 2,
Create = 3
};
enum Mode : u8 {
Read = 1,
Write = 2,
Create = 3
};
/**
/**
Opens a file
@param path The path to the file to open
@param mode File open mode
@return Handle to the newly opened file
@param path The path to the file to open
@param mode File open mode
@return Handle to the newly opened file
*/
fn open(str path, Mode mode) {
return builtin::std::file::open(path, u32(mode));
};
fn open(str path, Mode mode) {
return builtin::std::file::open(path, u32(mode));
};
/**
/**
Closes a file handle that has been opened previously
@param handle The handle to close
@param handle The handle to close
*/
fn close(Handle handle) {
builtin::std::file::close(handle);
};
fn close(Handle handle) {
builtin::std::file::close(handle);
};
/**
/**
Reads the content of a file into a string
@param handle The file handle to read from
@param size Number of bytes to read
@return String containing the read data
@param handle The file handle to read from
@param size Number of bytes to read
@return String containing the read data
*/
fn read(Handle handle, u64 size) {
return builtin::std::file::read(handle, size);
};
fn read(Handle handle, u64 size) {
return builtin::std::file::read(handle, size);
};
/**
/**
Writes the content of a string into a file
@param handle The file handle to write to
@param data String or Pattern to write to the file
@param handle The file handle to write to
@param data String or Pattern to write to the file
*/
fn write(Handle handle, auto data) {
builtin::std::file::write(handle, data);
};
fn write(Handle handle, auto data) {
builtin::std::file::write(handle, data);
};
/**
/**
Sets the current cursor position in the given file handle
@param handle The file handle to set the cursor position in
@param offset The offset to move the cursor to
@param handle The file handle to set the cursor position in
@param offset The offset to move the cursor to
*/
fn seek(Handle handle, u64 offset) {
builtin::std::file::seek(handle, offset);
};
fn seek(Handle handle, u64 offset) {
builtin::std::file::seek(handle, offset);
};
/**
Queries the size of a file
@param handle The handle of the file to get the size of
@return The file's size
*/
fn size(Handle handle) {
return builtin::std::file::size(handle);
};
/**
Queries the size of a file
@param handle The handle of the file to get the size of
@return The file's size
*/
fn size(Handle handle) {
return builtin::std::file::size(handle);
};
/**
Resizes a file
@param handle The handle of the file to resize
*/
fn resize(Handle handle, u64 size) {
builtin::std::file::resize(handle, size);
};
/**
Resizes a file
@param handle The handle of the file to resize
*/
fn resize(Handle handle, u64 size) {
builtin::std::file::resize(handle, size);
};
/**
Flushes changes made to a file to disk
@param handle The handle of the file to flush
*/
fn flush(Handle handle) {
builtin::std::file::remove(handle);
};
/**
Flushes changes made to a file to disk
@param handle The handle of the file to flush
*/
fn flush(Handle handle) {
builtin::std::file::flush(handle);
};
/**
Deletes a file from disk. This will also automatically close this file
@param handle The handle of the file to delete
*/
fn remove(Handle handle) {
builtin::std::file::remove(handle);
};
/**
Deletes a file from disk. This will also automatically close this file
@param handle The handle of the file to delete
*/
fn remove(Handle handle) {
builtin::std::file::remove(handle);
};
/**
Create all directories for the provided path
@param path The path for which all directories should be created
*/
fn create_directories(str path) {
builtin::std::file::create_directories(path);
};
}

View File

@@ -1,89 +1,89 @@
#pragma once
/*!
Library for doing arithmetic with fixed point numbers and converting them from/to floating point numbers.
Library for doing arithmetic with fixed point numbers and converting them from/to floating point numbers.
*/
namespace auto std::fxpt {
/**
A fixed point value
*/
using fixed = s128;
/**
A fixed point value
*/
using fixed = s128;
/**
Converts a fixed point value into a floating point value
@param fxt The fixed point value to convert
@param precision The bits of precision the new value should have
@return The floating point representation of fxt
*/
fn to_float(fixed fxt, u32 precision) {
return double(fxt) / double((1 << precision));
};
/**
Converts a floating point value into a fixed point value
@param flt The floating point value to convert
@param precision The bits of precision the new value should have
@return The fixed point representation of flt
*/
fn to_fixed(double flt, u32 precision) {
return s128((flt * (1 << precision)));
};
/**
Converts a fixed point value into a floating point value
@param fxt The fixed point value to convert
@param precision The bits of precision the new value should have
@return The floating point representation of fxt
*/
fn to_float(fixed fxt, u32 precision) {
return double(fxt) / double((1 << precision));
};
/**
Converts a floating point value into a fixed point value
@param flt The floating point value to convert
@param precision The bits of precision the new value should have
@return The fixed point representation of flt
*/
fn to_fixed(double flt, u32 precision) {
return s128((flt * (1 << precision)));
};
/**
Changes the number of bits used to represent the decimal part of the given fixed point number
@param value The fixed point value to convert
@param start_precision The current number of bits used
@param end_precision The new number of bits used
@return `value` as a new fixed point number with `end_precision` bits of precision
*/
fn change_precision(fixed value, u32 start_precision, u32 end_precision) {
return std::fxpt::to_fixed(std::fxpt::to_float(value, start_precision), end_precision);
};
/**
Adds two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the addition of `a` and `b`
*/
fn add(fixed a, fixed b, u32 precision) {
return a + b;
};
/**
Subtracts two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the subtraction of `a` and `b`
*/
fn subtract(fixed a, fixed b, u32 precision) {
return a - b;
};
/**
Multiplies two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the multiplication of `a` and `b`
*/
fn multiply(fixed a, fixed b, u32 precision) {
return (a * b) / (1 << precision);
};
/**
Divides two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the division of `a` and `b`
*/
fn divide(fixed a, fixed b, u32 precision) {
return (a << precision) / b;
};
/**
Changes the number of bits used to represent the decimal part of the given fixed point number
@param value The fixed point value to convert
@param start_precision The current number of bits used
@param end_precision The new number of bits used
@return `value` as a new fixed point number with `end_precision` bits of precision
*/
fn change_precision(fixed value, u32 start_precision, u32 end_precision) {
return std::fxpt::to_fixed(std::fxpt::to_float(value, start_precision), end_precision);
};
/**
Adds two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the addition of `a` and `b`
*/
fn add(fixed a, fixed b, u32 precision) {
return a + b;
};
/**
Subtracts two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the subtraction of `a` and `b`
*/
fn subtract(fixed a, fixed b, u32 precision) {
return a - b;
};
/**
Multiplies two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the multiplication of `a` and `b`
*/
fn multiply(fixed a, fixed b, u32 precision) {
return (a * b) / (1 << precision);
};
/**
Divides two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the division of `a` and `b`
*/
fn divide(fixed a, fixed b, u32 precision) {
return (a << precision) / b;
};
}

View File

@@ -6,9 +6,37 @@
namespace auto std::hash {
/**
Calculates the CRC8 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC8 hash of
@param init The CRC8 init value
@param poly The CRC8 polynomial
@param xorout The CRC8 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC8 hash
*/
fn crc8(ref auto pattern, u8 init, u8 poly, u8 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc8(pattern, init, poly, xorout, reflect_in, reflect_out);
};
/**
Calculates the CRC16 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC16 hash of
@param init The CRC16 init value
@param poly The CRC16 polynomial
@param xorout The CRC16 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC16 hash
*/
fn crc16(ref auto pattern, u16 init, u16 poly, u16 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc16(pattern, init, poly, xorout, reflect_in, reflect_out);
};
/**
Calculates the CRC32 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the crc32 hash of
@param pattern The pattern to calculate the CRC32 hash of
@param init The CRC32 init value
@param poly The CRC32 polynomial
@param xorout The CRC32 XOR-Out value
@@ -20,4 +48,18 @@ namespace auto std::hash {
return builtin::std::hash::crc32(pattern, init, poly, xorout, reflect_in, reflect_out);
};
/**
Calculates the CRC64 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC64 hash of
@param init The CRC64 init value
@param poly The CRC64 polynomial
@param xorout The CRC64 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC64 hash
*/
fn crc64(ref auto pattern, u64 init, u64 poly, u64 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc64(pattern, init, poly, xorout, reflect_in, reflect_out);
};
}

View File

@@ -1,169 +1,169 @@
#pragma once
/*!
Library to calculate the minimum and maximum values that fit into a given data type
Library to calculate the minimum and maximum values that fit into a given data type
*/
namespace auto std::limits {
/**
Returns the minimum value that can be stored in a `u8`.
@return Minimum value
*/
fn u8_min() {
return u8(0);
};
/**
Returns the maximum value that can be stored in a `u8`.
@return Maximum value
*/
fn u8_max() {
return u8(-1);
};
/**
Returns the minimum value that can be stored in a `u8`.
@return Minimum value
*/
fn u8_min() {
return u8(0);
};
/**
Returns the maximum value that can be stored in a `u8`.
@return Maximum value
*/
fn u8_max() {
return u8(-1);
};
/**
Returns the minimum value that can be stored in a `s8`.
@return Minimum value
*/
/**
Returns the minimum value that can be stored in a `s8`.
@return Minimum value
*/
fn s8_min() {
return -s8((std::limits::u8_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s8`.
@return Maximum value
*/
fn s8_max() {
return s8((std::limits::u8_max() / 2));
};
return -s8((std::limits::u8_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s8`.
@return Maximum value
*/
fn s8_max() {
return s8((std::limits::u8_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u16`.
@return Minimum value
*/
fn u16_min() {
return u16(0);
};
/**
Returns the maximum value that can be stored in a `u16`.
@return Maximum value
*/
fn u16_max() {
return u16(-1);
};
/**
Returns the minimum value that can be stored in a `s16`.
@return Minimum value
*/
fn s16_min() {
return -s16((std::limits::u16_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s16`.
@return Maximum value
*/
fn s16_max() {
return s16((std::limits::u16_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u32`.
@return Minimum value
*/
fn u32_min() {
return u32(0);
};
/**
Returns the maximum value that can be stored in a `u32`.
@return Maximum value
*/
fn u32_max() {
return u32(-1);
};
/**
Returns the minimum value that can be stored in a `s32`.
@return Minimum value
*/
fn s32_min() {
return -s32((std::limits::u32_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s32`.
@return Maximum value
*/
fn s32_max() {
return s32((std::limits::u32_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u64`.
@return Minimum value
*/
fn u64_min() {
return u64(0);
};
/**
Returns the maximum value that can be stored in a `u64`.
@return Maximum value
*/
fn u64_max() {
return u64(-1);
};
/**
Returns the minimum value that can be stored in a `s64`.
@return Minimum value
*/
fn s64_min() {
return -s64((std::limits::u64_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s64`.
@return Maximum value
*/
fn s64_max() {
return s64((std::limits::u64_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u128`.
@return Minimum value
*/
fn u128_min() {
return u128(0);
};
/**
Returns the maximum value that can be stored in a `u128`.
@return Maximum value
*/
fn u128_max() {
return u128(-1);
};
/**
Returns the minimum value that can be stored in a `s128`.
@return Minimum value
*/
fn s128_min() {
return -s128((std::limits::u128_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s128`.
@return Maximum value
*/
fn s128_max() {
return s128((std::limits::u128_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u16`.
@return Minimum value
*/
fn u16_min() {
return u16(0);
};
/**
Returns the maximum value that can be stored in a `u16`.
@return Maximum value
*/
fn u16_max() {
return u16(-1);
};
/**
Returns the minimum value that can be stored in a `s16`.
@return Minimum value
*/
fn s16_min() {
return -s16((std::limits::u16_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s16`.
@return Maximum value
*/
fn s16_max() {
return s16((std::limits::u16_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u32`.
@return Minimum value
*/
fn u32_min() {
return u32(0);
};
/**
Returns the maximum value that can be stored in a `u32`.
@return Maximum value
*/
fn u32_max() {
return u32(-1);
};
/**
Returns the minimum value that can be stored in a `s32`.
@return Minimum value
*/
fn s32_min() {
return -s32((std::limits::u32_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s32`.
@return Maximum value
*/
fn s32_max() {
return s32((std::limits::u32_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u64`.
@return Minimum value
*/
fn u64_min() {
return u64(0);
};
/**
Returns the maximum value that can be stored in a `u64`.
@return Maximum value
*/
fn u64_max() {
return u64(-1);
};
/**
Returns the minimum value that can be stored in a `s64`.
@return Minimum value
*/
fn s64_min() {
return -s64((std::limits::u64_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s64`.
@return Maximum value
*/
fn s64_max() {
return s64((std::limits::u64_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u128`.
@return Minimum value
*/
fn u128_min() {
return u128(0);
};
/**
Returns the maximum value that can be stored in a `u128`.
@return Maximum value
*/
fn u128_max() {
return u128(-1);
};
/**
Returns the minimum value that can be stored in a `s128`.
@return Minimum value
*/
fn s128_min() {
return -s128((std::limits::u128_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s128`.
@return Maximum value
*/
fn s128_max() {
return s128((std::limits::u128_max() / 2));
};
}

View File

@@ -3,338 +3,338 @@
import std.mem;
/*!
Library containing more advanced mathematical operations.
Library containing more advanced mathematical operations.
*/
namespace auto std::math {
/**
Compares the values `a` and `b` with each other and returns the smaller of the two
@param a First value
@param b Second value
@return `a` if `a` is smaller than `b`, otherwise `b`
*/
fn min(auto a, auto b) {
if (a < b)
return a;
else
return b;
};
/**
Compares the values `a` and `b` with each other and returns the bigger of the two
@param a First value
@param b Second value
@return `a` if `a` is bigger than `b`, otherwise `b`
*/
fn max(auto a, auto b) {
if (a > b)
return a;
else
return b;
};
/**
Clamps the value of `x` between `min` and `max`.
@param x Value
@param min Minimum value
@param max Maximum value
@return `min` if `x` is smaller than `min`, `max` if `x` is bigger than `max`, `x` otherwise
*/
fn clamp(auto x, auto min, auto max) {
if (x < min)
return min;
else if (x > max)
return max;
else
return x;
};
/**
Returns the absolute value of `x`.
@param x Value
@return `x` if `x` is positive, `-x` otherwise
*/
fn abs(auto x) {
if (x < 0)
return -x;
else
return x;
};
/**
Returns the sign of `x`.
@param x Value
@return `1` if `x` is positive, `-1` if `x` is negative, `0` if `x` is zero
*/
fn sign(auto x) {
if (x > 0)
return 1;
else if (x < 0)
return -1;
else
return 0;
};
/**
Copies the sign of `y` to `x`.
@param x Value
@param y Value
@return `x` if `y` is positive, `-x` if `y` is negative
*/
fn copy_sign(auto x, auto y) {
if (y >= 0)
return std::math::abs(x);
else
return -std::math::abs(x);
};
/**
Calculates the factorial of `x`.
@param x Value
@return Factorial of `x`
*/
fn factorial(u128 x) {
u128 result;
result = x;
while (x > 1) {
x = x - 1;
result = result * x;
}
return result;
};
/**
Calculates the binomial coefficient of `n` and `k`.
@param n Value
@param k Value
@return Binomial coefficient of `n` and `k`
*/
fn comb(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / (std::math::factorial(k) * std::math::factorial(n - k));
};
/**
Calculates the permutation of `n` and `k`.
@param n Value
@param k Value
@return Permutation of `n` and `k`
*/
fn perm(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / std::math::factorial(n - k);
};
/**
Floors the value of `value`.
@param value Value
@return `value` floored
*/
fn floor(auto value) { return builtin::std::math::floor(value); };
/**
Ceils the value of `value`.
@param value Value
@return `value` ceiled
*/
fn ceil(auto value) { return builtin::std::math::ceil(value); };
/**
Rounds the value of `value`.
@param value Value
@return `value` rounded
*/
fn round(auto value) { return builtin::std::math::round(value); };
/**
Truncates the value of `value`.
@param value Value
@return `value` truncated
*/
fn trunc(auto value) { return builtin::std::math::trunc(value); };
/**
Calculates the logarithm of `value` with base 10.
@param value Value
@return Logarithm of `value` with base 10
*/
fn log10(auto value) { return builtin::std::math::log10(value); };
/**
Calculates the logarithm of `value` with base 2.
@param value Value
@return Logarithm of `value` with base 2
*/
fn log2(auto value) { return builtin::std::math::log2(value); };
/**
Calculates the natural logarithm of `value`.
@param value Value
@return Logarithm of `value` with base `e`
*/
fn ln(auto value) { return builtin::std::math::ln(value); };
/**
Calculates the floating point modulus of `value`.
@param value Value
@return Floating point modulus of `value`
*/
fn fmod(auto value) { return builtin::std::math::fmod(value); };
/**
Calculates the value of `base` raised to the power of `exp`.
@param base Base
@param exp Exponent
@return `base` raised to the power of `exp`
*/
fn pow(auto base, auto exp) { return builtin::std::math::pow(base, exp); };
/**
Compares the values `a` and `b` with each other and returns the smaller of the two
@param a First value
@param b Second value
@return `a` if `a` is smaller than `b`, otherwise `b`
*/
fn min(auto a, auto b) {
if (a < b)
return a;
else
return b;
};
/**
Compares the values `a` and `b` with each other and returns the bigger of the two
@param a First value
@param b Second value
@return `a` if `a` is bigger than `b`, otherwise `b`
*/
fn max(auto a, auto b) {
if (a > b)
return a;
else
return b;
};
/**
Calculates the value of the natural number `e` raised to the power of `value`.
@param value Exponent
@return `e` raised to the power of `value`
*/
Clamps the value of `x` between `min` and `max`.
@param x Value
@param min Minimum value
@param max Maximum value
@return `min` if `x` is smaller than `min`, `max` if `x` is bigger than `max`, `x` otherwise
*/
fn clamp(auto x, auto min, auto max) {
if (x < min)
return min;
else if (x > max)
return max;
else
return x;
};
/**
Returns the absolute value of `x`.
@param x Value
@return `x` if `x` is positive, `-x` otherwise
*/
fn abs(auto x) {
if (x < 0)
return -x;
else
return x;
};
/**
Returns the sign of `x`.
@param x Value
@return `1` if `x` is positive, `-1` if `x` is negative, `0` if `x` is zero
*/
fn sign(auto x) {
if (x > 0)
return 1;
else if (x < 0)
return -1;
else
return 0;
};
/**
Copies the sign of `y` to `x`.
@param x Value
@param y Value
@return `x` if `y` is positive, `-x` if `y` is negative
*/
fn copy_sign(auto x, auto y) {
if (y >= 0)
return std::math::abs(x);
else
return -std::math::abs(x);
};
/**
Calculates the factorial of `x`.
@param x Value
@return Factorial of `x`
*/
fn factorial(u128 x) {
u128 result;
result = x;
while (x > 1) {
x = x - 1;
result = result * x;
}
return result;
};
/**
Calculates the binomial coefficient of `n` and `k`.
@param n Value
@param k Value
@return Binomial coefficient of `n` and `k`
*/
fn comb(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / (std::math::factorial(k) * std::math::factorial(n - k));
};
/**
Calculates the permutation of `n` and `k`.
@param n Value
@param k Value
@return Permutation of `n` and `k`
*/
fn perm(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / std::math::factorial(n - k);
};
/**
Floors the value of `value`.
@param value Value
@return `value` floored
*/
fn floor(auto value) { return builtin::std::math::floor(value); };
/**
Ceils the value of `value`.
@param value Value
@return `value` ceiled
*/
fn ceil(auto value) { return builtin::std::math::ceil(value); };
/**
Rounds the value of `value`.
@param value Value
@return `value` rounded
*/
fn round(auto value) { return builtin::std::math::round(value); };
/**
Truncates the value of `value`.
@param value Value
@return `value` truncated
*/
fn trunc(auto value) { return builtin::std::math::trunc(value); };
/**
Calculates the logarithm of `value` with base 10.
@param value Value
@return Logarithm of `value` with base 10
*/
fn log10(auto value) { return builtin::std::math::log10(value); };
/**
Calculates the logarithm of `value` with base 2.
@param value Value
@return Logarithm of `value` with base 2
*/
fn log2(auto value) { return builtin::std::math::log2(value); };
/**
Calculates the natural logarithm of `value`.
@param value Value
@return Logarithm of `value` with base `e`
*/
fn ln(auto value) { return builtin::std::math::ln(value); };
/**
Calculates the floating point modulus of `value`.
@param value Value
@return Floating point modulus of `value`
*/
fn fmod(auto value) { return builtin::std::math::fmod(value); };
/**
Calculates the value of `base` raised to the power of `exp`.
@param base Base
@param exp Exponent
@return `base` raised to the power of `exp`
*/
fn pow(auto base, auto exp) { return builtin::std::math::pow(base, exp); };
/**
Calculates the value of the natural number `e` raised to the power of `value`.
@param value Exponent
@return `e` raised to the power of `value`
*/
fn exp(auto value) { return builtin::std::math::exp(value); };
/**
Calculates the square root of `value`.
@param value Value
@return Square root of `value`
*/
fn sqrt(auto value) { return builtin::std::math::sqrt(value); };
/**
Calculates the square root of `value`.
@param value Value
@return Square root of `value`
*/
fn sqrt(auto value) { return builtin::std::math::sqrt(value); };
/**
Calculates the cubic root of `value`.
@param value Value
@return Cubic root of `value`
*/
fn cbrt(auto value) { return builtin::std::math::cbrt(value); };
/**
Calculates the cubic root of `value`.
@param value Value
@return Cubic root of `value`
*/
fn cbrt(auto value) { return builtin::std::math::cbrt(value); };
/**
Calculates the sine of `value`.
@param value Angle value in radians
@return Sine of `value`
*/
fn sin(auto value) { return builtin::std::math::sin(value); };
/**
Calculates the sine of `value`.
@param value Angle value in radians
@return Sine of `value`
*/
fn sin(auto value) { return builtin::std::math::sin(value); };
/**
Calculates the cosine of `value`.
@param value Angle value in radians
@return Cosine of `value`
*/
fn cos(auto value) { return builtin::std::math::cos(value); };
/**
Calculates the cosine of `value`.
@param value Angle value in radians
@return Cosine of `value`
*/
fn cos(auto value) { return builtin::std::math::cos(value); };
/**
Calculates the tangent of `value`.
@param value Angle value in radians
@return Tangent of `value`
*/
fn tan(auto value) { return builtin::std::math::tan(value); };
/**
Calculates the tangent of `value`.
@param value Angle value in radians
@return Tangent of `value`
*/
fn tan(auto value) { return builtin::std::math::tan(value); };
/**
Calculates the arc sine of `value`.
@param value Angle value in radians
@return Arc sine of `value`
*/
fn asin(auto value) { return builtin::std::math::asin(value); };
/**
Calculates the arc sine of `value`.
@param value Angle value in radians
@return Arc sine of `value`
*/
fn asin(auto value) { return builtin::std::math::asin(value); };
/**
Calculates the arc cosine of `value`.
@param value Value
@return Arc cosine of `value` in radians
*/
fn acos(auto value) { return builtin::std::math::acos(value); };
/**
Calculates the arc cosine of `value`.
@param value Value
@return Arc cosine of `value` in radians
*/
fn acos(auto value) { return builtin::std::math::acos(value); };
/**
Calculates the arc tangent of `value`.
@param value Value
@return Arc tangent of `value` in radians between `-pi/2` and `pi/2`
*/
fn atan(auto value) { return builtin::std::math::atan(value); };
/**
Calculates the arc tangent of `value`.
@param value Value
@return Arc tangent of `value` in radians between `-pi/2` and `pi/2`
*/
fn atan(auto value) { return builtin::std::math::atan(value); };
/**
Calculates the arc tangent of `value`.
@param y Value representing the proportion of the y-coordinate
@param x Value representing the proportion of the x-coordinate.
@return Arc tangent of `value` in radians between `-pi` and `pi`
*/
fn atan2(auto y, auto x) { return builtin::std::math::atan2(y, x); };
/**
Calculates the arc tangent of `value`.
@param y Value representing the proportion of the y-coordinate
@param x Value representing the proportion of the x-coordinate.
@return Arc tangent of `value` in radians between `-pi` and `pi`
*/
fn atan2(auto y, auto x) { return builtin::std::math::atan2(y, x); };
/**
Calculates the hyperbolic sine of `value`.
@param value Angle value in radians
@return Hyperbolic sine of `value`
*/
fn sinh(auto value) { return builtin::std::math::sinh(value); };
/**
Calculates the hyperbolic sine of `value`.
@param value Angle value in radians
@return Hyperbolic sine of `value`
*/
fn sinh(auto value) { return builtin::std::math::sinh(value); };
/**
Calculates the hyperbolic cosine of `value`.
@param value Angle value in radians
@return Hyperbolic cosine of `value`
*/
fn cosh(auto value) { return builtin::std::math::cosh(value); };
/**
Calculates the hyperbolic cosine of `value`.
@param value Angle value in radians
@return Hyperbolic cosine of `value`
*/
fn cosh(auto value) { return builtin::std::math::cosh(value); };
/**
Calculates the hyperbolic tangent of `value`.
@param value Angle value in radians
@return Hyperbolic tangent of `value`
*/
fn tanh(auto value) { return builtin::std::math::tanh(value); };
/**
Calculates the hyperbolic tangent of `value`.
@param value Angle value in radians
@return Hyperbolic tangent of `value`
*/
fn tanh(auto value) { return builtin::std::math::tanh(value); };
/**
Calculates the arc hyperbolic sine of `value`.
@param value Value
@return Arc hyperbolic sine of `value`
*/
fn asinh(auto value) { return builtin::std::math::asinh(value); };
/**
Calculates the arc hyperbolic sine of `value`.
@param value Value
@return Arc hyperbolic sine of `value`
*/
fn asinh(auto value) { return builtin::std::math::asinh(value); };
/**
Calculates the arc hyperbolic cosine of `value`.
@param value Value
@return Arc hyperbolic cosine of `value`
*/
fn acosh(auto value) { return builtin::std::math::acosh(value); };
/**
Calculates the arc hyperbolic cosine of `value`.
@param value Value
@return Arc hyperbolic cosine of `value`
*/
fn acosh(auto value) { return builtin::std::math::acosh(value); };
/**
Calculates the arc hyperbolic tangent of `value`.
@param value Value
@return Arc hyperbolic tangent of `value`
*/
fn atanh(auto value) { return builtin::std::math::atanh(value); };
/**
Calculates the arc hyperbolic tangent of `value`.
@param value Value
@return Arc hyperbolic tangent of `value`
*/
fn atanh(auto value) { return builtin::std::math::atanh(value); };
/**
Options to use with the `std::math::accumulate` function.
*/
enum AccumulateOperation : u8 {
Add = 0,
Multiply = 1,
Modulo = 2,
Min = 3,
Max = 4
};
/**
Options to use with the `std::math::accumulate` function.
*/
enum AccumulateOperation : u8 {
Add = 0,
Multiply = 1,
Modulo = 2,
Min = 3,
Max = 4
};
/**
Calculates the sum of all values in the specified memory range.
@param start Start address
@param end End address
@param valueSize Size of each value in bytes
@param [section] Section to use
@param [operation] Operation to use. Defaults to addition
@param [endian] Endianness to use. Defaults to native
@return Sum of all values in the specified memory range
*/
fn accumulate(u128 start, u128 end, u128 valueSize, std::mem::Section section = 0, AccumulateOperation operation = AccumulateOperation::Add, std::mem::Endian endian = std::mem::Endian::Native) {
return builtin::std::math::accumulate(start, end, valueSize, section, u128(operation), u128(endian));
};
/**
Calculates the sum of all values in the specified memory range.
@param start Start address
@param end End address
@param valueSize Size of each value in bytes
@param [section] Section to use
@param [operation] Operation to use. Defaults to addition
@param [endian] Endianness to use. Defaults to native
@return Sum of all values in the specified memory range
*/
fn accumulate(u128 start, u128 end, u128 valueSize, std::mem::Section section = 0, AccumulateOperation operation = AccumulateOperation::Add, std::mem::Endian endian = std::mem::Endian::Native) {
return builtin::std::math::accumulate(start, end, valueSize, section, u128(operation), u128(endian));
};
}

View File

@@ -9,18 +9,15 @@ namespace auto std::mem {
namespace impl {
struct MagicSearchImpl<auto Magic, T> {
if ($ < (std::mem::base_address() + std::mem::size() - std::string::length(Magic) - 1)) {
char __potentialMagic__[std::string::length(Magic)] [[hidden, no_unique_address]];
if (__potentialMagic__ == Magic) {
T data [[inline]];
} else {
padding[1];
continue;
}
} else {
padding[1];
continue;
s128 address = builtin::std::mem::find_string_in_range(0, $, builtin::std::mem::size(), Magic);
if (address < 0)
break;
$ = address;
try {
T data [[inline]];
} catch {
T data;
}
};
@@ -63,7 +60,7 @@ namespace auto std::mem {
/**
Gets the base address of the memory
Gets the base address of the data
@return The base address of the memory
*/
fn base_address() {
@@ -71,7 +68,7 @@ namespace auto std::mem {
};
/**
Gets the size of the memory
Gets the size of the data
@return The size of the memory
*/
fn size() {
@@ -79,17 +76,18 @@ namespace auto std::mem {
};
/**
Finds a sequence of bytes in the memory
Finds a sequence of bytes in the data
@param occurrence_index The index of the occurrence to find
@param bytes The bytes to find
@return The address of the sequence
*/
fn find_sequence(u128 occurrence_index, auto ... bytes) {
return builtin::std::mem::find_sequence_in_range(occurrence_index, builtin::std::mem::base_address(), builtin::std::mem::size(), bytes);
const u128 address = builtin::std::mem::base_address();
return builtin::std::mem::find_sequence_in_range(occurrence_index, address, address + builtin::std::mem::size(), bytes);
};
/**
Finds a sequence of bytes in a specific region of the memory
Finds a sequence of bytes in a specific region of the data
@param occurrence_index The index of the occurrence to find
@param offsetFrom The offset from which to start searching
@param offsetTo The offset to which to search
@@ -100,6 +98,30 @@ namespace auto std::mem {
return builtin::std::mem::find_sequence_in_range(occurrence_index, offsetFrom, offsetTo, bytes);
};
/**
Finds a string in the data
@param occurrence_index The index of the occurrence to find
@param string The string to find
@return The address of the sequence
*/
fn find_string(u128 occurrence_index, str string) {
const u128 address = builtin::std::mem::base_address();
return builtin::std::mem::find_string_in_range(occurrence_index, address, address + builtin::std::mem::size(), string);
};
/**
Finds a string in a specific region of the data
@param occurrence_index The index of the occurrence to find
@param offsetFrom The offset from which to start searching
@param offsetTo The offset to which to search
@param string The string to find
@return The address of the sequence
*/
fn find_string_in_range(u128 occurrence_index, u128 offsetFrom, u128 offsetTo, str string) {
return builtin::std::mem::find_string_in_range(occurrence_index, offsetFrom, offsetTo, string);
};
/**
Reads a unsigned value from the memory
@param address The address to read from
@@ -132,14 +154,6 @@ namespace auto std::mem {
return builtin::std::mem::read_string(address, size);
};
/**
Gets the current bit offset within the current byte that a bitfield will read.
*/
fn current_bit_offset() {
return builtin::std::mem::current_bit_offset();
};
/**
Reads a number of bits from the specified bit offset within the specified byte
@param byteOffset The byte offset within the data
@@ -211,6 +225,14 @@ namespace auto std::mem {
builtin::std::mem::copy_value_to_section(value, to_section, to_address);
};
/**
Returns the current bit offset when inside of a bitfield.
@return The current bit offset between 0 and 7
*/
fn current_bit_offset() {
return builtin::std::mem::current_bit_offset();
};
/**
Searches for a sequence of bytes and places the given type at that address

View File

@@ -1,129 +1,149 @@
#pragma once
import std.io;
import std.mem;
/*!
Libray to interact with strings.
Libray to interact with strings.
*/
namespace auto std::string {
/**
Base type for sized strings. Represents a string with its size preceeding it.
@tparam SizeType The type of the size field.
@tparam DataType The type of the characters.
*/
struct SizedStringBase<SizeType, DataType> {
SizeType size;
DataType data[size];
} [[sealed, format("std::string::impl::format_sized_string"), transform("std::string::impl::format_sized_string")]];
/**
Base type for sized strings. Represents a string with its size preceeding it.
@tparam SizeType The type of the size field.
@tparam DataType The type of the characters.
*/
struct SizedStringBase<SizeType, DataType> {
SizeType size;
DataType data[size];
} [[sealed, format("std::string::impl::format_string"), transform("std::string::impl::format_string")]];
/**
A ASCII string with a prefixed size.
@tparam SizeType The type of the size field.
*/
using SizedString<SizeType> = SizedStringBase<SizeType, char>;
/**
A ASCII string with a prefixed size.
@tparam SizeType The type of the size field.
*/
using SizedString<SizeType> = SizedStringBase<SizeType, char>;
/**
A UTF-16 string with a prefixed size.
@tparam SizeType The type of the size field.
*/
using SizedString16<SizeType> = SizedStringBase<SizeType, char16>;
/**
A UTF-16 string with a prefixed size.
@tparam SizeType The type of the size field.
*/
using SizedString16<SizeType> = SizedStringBase<SizeType, char16>;
namespace impl {
/**
Base type for null-terminated strings. Represents a string with its size determined by the first 0x00 byte found.
@tparam DataType The type of the characters.
*/
struct NullStringBase<DataType> {
DataType string[while(std::mem::read_unsigned($, sizeof(DataType)) != 0x00)];
DataType null_terminator;
} [[sealed, format("std::string::impl::format_string"), transform("std::string::impl::format_string")]];
fn format_sized_string(ref auto string) {
return string.data;
};
/**
A null-terminated ASCII string.
*/
using NullString = NullStringBase<char>;
}
/**
A null-terminated UTF-16 string.
*/
using NullString16 = NullStringBase<char16>;
/**
Gets the length of a string.
@param string The string to get the length of.
@return The length of the string.
*/
fn length(str string) {
return builtin::std::string::length(string);
};
namespace impl {
/**
Gets the character at a given index.
@param string The string to get the character from.
@param index The index of the character to get.
@return The character at the given index.
*/
fn at(str string, u32 index) {
return builtin::std::string::at(string, index);
};
fn format_string(ref auto string) {
return string.data;
};
/**
Gets a substring of a string.
@param string The string to get the substring from.
@param pos The position of the first character of the substring.
@param count The number of characters to get.
@return The substring.
*/
fn substr(str string, u32 pos, u32 count) {
return builtin::std::string::substr(string, pos, count);
};
}
/**
Gets the length of a string.
@param string The string to get the length of.
@return The length of the string.
*/
fn length(str string) {
return builtin::std::string::length(string);
};
/**
Gets the character at a given index.
@param string The string to get the character from.
@param index The index of the character to get.
@return The character at the given index.
*/
fn at(str string, u32 index) {
return builtin::std::string::at(string, index);
};
/**
Gets a substring of a string.
@param string The string to get the substring from.
@param pos The position of the first character of the substring.
@param count The number of characters to get.
@return The substring.
*/
fn substr(str string, u32 pos, u32 count) {
return builtin::std::string::substr(string, pos, count);
};
/**
Converts a string to an integer.
@param string The string to convert.
@param base The base of the number.
@return The integer.
*/
fn parse_int(str string, u8 base) {
return builtin::std::string::parse_int(string, base);
};
/**
Converts a string to an integer.
@param string The string to convert.
@param base The base of the number.
@return The integer.
*/
fn parse_int(str string, u8 base) {
return builtin::std::string::parse_int(string, base);
};
/**
Converts a string to a float.
@param string The string to convert.
@return The float.
*/
fn parse_float(str string) {
return builtin::std::string::parse_float(string);
};
/**
Converts a string to a float.
@param string The string to convert.
@return The float.
*/
fn parse_float(str string) {
return builtin::std::string::parse_float(string);
};
/**
Converts any type to a string.
@param x The value to convert.
@return The string.
*/
/**
Converts any type to a string.
@param x The value to convert.
@return The string.
*/
fn to_string(auto x) {
return std::format("{}", x);
};
/**
Checks if a string starts with a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string starts with the substring, false otherwise.
*/
/**
Checks if a string starts with a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string starts with the substring, false otherwise.
*/
fn starts_with(str string, str part) {
return std::string::substr(string, 0, std::string::length(part)) == part;
};
/**
Checks if a string ends with a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string ends with the substring, false otherwise.
*/
/**
Checks if a string ends with a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string ends with the substring, false otherwise.
*/
fn ends_with(str string, str part) {
return std::string::substr(string, std::string::length(string) - std::string::length(part), std::string::length(part)) == part;
};
/**
Checks if a string contains a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string contains the substring, false otherwise.
*/
/**
Checks if a string contains a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string contains the substring, false otherwise.
*/
fn contains(str string, str part) {
s32 string_len = std::string::length(string);
s32 part_len = std::string::length(part);
@@ -136,12 +156,12 @@ namespace auto std::string {
return false;
};
/**
Reverses a string.
@param string The string to reverse.
@return The reversed string.
*/
fn reverse(str string) {
/**
Reverses a string.
@param string The string to reverse.
@return The reversed string.
*/
fn reverse(str string) {
str result;
s32 i;
@@ -154,82 +174,82 @@ namespace auto std::string {
return result;
};
/**
Converts a string to upper case.
@param string The string to convert.
@return The converted string.
*/
/**
Converts a string to upper case.
@param string The string to convert.
@return The converted string.
*/
fn to_upper(str string) {
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'a' && c <= 'z')
result = result + char(c - 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
/**
Converts a string to lower case.
@param string The string to convert.
@return The converted string.
*/
fn to_lower(str string) {
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'A' && c <= 'Z')
result = result + char(c + 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'a' && c <= 'z')
result = result + char(c - 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
/**
Converts a string to lower case.
@param string The string to convert.
@return The converted string.
*/
fn to_lower(str string) {
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'A' && c <= 'Z')
result = result + char(c + 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
/**
Replaces all occurrences of a substring with another substring.
@param string The string to replace in.
@param pattern The substring to replace.
@param replace The substring to replace with.
@return The string with the replacements.
*/
fn replace(str string, str pattern, str replace) {
s32 string_len = std::string::length(string);
s32 pattern_len = std::string::length(pattern);
/**
Replaces all occurrences of a substring with another substring.
@param string The string to replace in.
@param pattern The substring to replace.
@param replace The substring to replace with.
@return The string with the replacements.
*/
fn replace(str string, str pattern, str replace) {
s32 string_len = std::string::length(string);
s32 pattern_len = std::string::length(pattern);
if (pattern_len > string_len || pattern_len * string_len == 0 )
return string;
if (pattern_len > string_len || pattern_len * string_len == 0 )
return string;
str result;
s32 string_index;
s32 remaining_len = string_len;
while (pattern_len <= remaining_len) {
if (std::string::substr(string, string_index, pattern_len) == pattern) {
result += replace;
string_index += pattern_len;
} else {
result += std::string::at(string, string_index);
string_index += 1;
}
remaining_len = string_len - string_index;
}
result += std::string::substr(string, string_index, remaining_len );
return result;
};
str result;
s32 string_index;
s32 remaining_len = string_len;
while (pattern_len <= remaining_len) {
if (std::string::substr(string, string_index, pattern_len) == pattern) {
result += replace;
string_index += pattern_len;
} else {
result += std::string::at(string, string_index);
string_index += 1;
}
remaining_len = string_len - string_index;
}
result += std::string::substr(string, string_index, remaining_len );
return result;
};
}

View File

@@ -10,7 +10,7 @@ namespace auto std {
/**
Asserts that a given value is true. If it's not, abort evaluation and print the given message to the console
@param conditoon The condition that is required to be true
@param condition The condition that is required to be true
@param message The message to print in case the assertion doesn't hold
*/
fn assert(bool condition, str message) {
@@ -21,7 +21,7 @@ namespace auto std {
/**
Asserts that a given value is true. If it's not, print the given message to the console as a warning
@param conditoon The condition that is required to be true
@param condition The condition that is required to be true
@param message The message to print in case the assertion doesn't hold
*/
fn assert_warn(bool condition, str message) {
@@ -49,4 +49,11 @@ namespace auto std {
return builtin::std::sizeof_pack(pack);
};
/**
Throws an error notifying the developer that the current codepath is not implemented currently.
*/
fn unimplemented() {
std::error("Unimplemented code path reached!");
};
}

View File

@@ -3,198 +3,198 @@
import std.io;
/*!
Library to handle time and date related operations.
Library to handle time and date related operations.
*/
namespace auto std::time {
/**
A structured representation of a time and date.
*/
struct Time {
u8 sec;
u8 min;
u8 hour;
u8 mday;
u8 mon;
u16 year;
u8 wday;
u16 yday;
bool isdst;
} [[sealed]];
/**
A structured representation of a time and date.
*/
struct Time {
u8 sec;
u8 min;
u8 hour;
u8 mday;
u8 mon;
u16 year;
u8 wday;
u16 yday;
bool isdst;
} [[sealed]];
/**
A helper type to convert between Time and u128.
*/
union TimeConverter {
Time time;
u128 value;
};
/**
A helper type to convert between Time and u128.
*/
union TimeConverter {
Time time;
u128 value;
};
/**
A type to represent a time in seconds since the epoch.
*/
using EpochTime = u32;
/**
A type to represent a time in seconds since the epoch.
*/
using EpochTime = u32;
/**
A type to represent a time zone.
*/
enum TimeZone : u8 {
Local,
UTC
};
/**
A type to represent a time zone.
*/
enum TimeZone : u8 {
Local,
UTC
};
/**
A type to represent a DOS date.
*/
bitfield DOSDate {
day: 5;
month: 4;
year: 7;
} [[sealed]];
/**
A type to represent a DOS date.
*/
bitfield DOSDate {
day: 5;
month: 4;
year: 7;
} [[sealed]];
/**
A type to represent a DOS time.
*/
bitfield DOSTime {
seconds: 5;
minutes: 6;
hours: 5;
} [[sealed]];
/**
A type to represent a DOS time.
*/
bitfield DOSTime {
seconds: 5;
minutes: 6;
hours: 5;
} [[sealed]];
namespace impl {
namespace impl {
union DOSDateConverter {
DOSDate date;
u16 value;
};
union DOSDateConverter {
DOSDate date;
u16 value;
};
union DOSTimeConverter {
DOSTime time;
u16 value;
};
union DOSTimeConverter {
DOSTime time;
u16 value;
};
}
}
/**
Returns the current time in seconds since the epoch.
@return The current time in seconds since the epoch.
*/
fn epoch() {
return builtin::std::time::epoch();
};
/**
Returns the current time in seconds since the epoch.
@return The current time in seconds since the epoch.
*/
fn epoch() {
return builtin::std::time::epoch();
};
/**
Converts a time in seconds since the epoch to a local time.
@param epoch_time The time in seconds since the epoch.
@return The local time.
*/
fn to_local(EpochTime epoch_time) {
le TimeConverter converter;
/**
Converts a time in seconds since the epoch to a local time.
@param epoch_time The time in seconds since the epoch.
@return The local time.
*/
fn to_local(EpochTime epoch_time) {
le TimeConverter converter;
converter.value = builtin::std::time::to_local(epoch_time);
converter.value = builtin::std::time::to_local(epoch_time);
return converter.time;
};
return converter.time;
};
/**
Converts a time in seconds since the epoch to a UTC time.
@param epoch_time The time in seconds since the epoch.
@return The UTC time.
*/
fn to_utc(EpochTime epoch_time) {
le TimeConverter converter;
/**
Converts a time in seconds since the epoch to a UTC time.
@param epoch_time The time in seconds since the epoch.
@return The UTC time.
*/
fn to_utc(EpochTime epoch_time) {
le TimeConverter converter;
converter.value = builtin::std::time::to_utc(epoch_time);
converter.value = builtin::std::time::to_utc(epoch_time);
return converter.time;
};
return converter.time;
};
/**
Queries the current time in the specified time zone.
@param [time_zone] The time zone to query. Defaults to local.
@return The current time in the specified time zone.
*/
fn now(TimeZone time_zone = TimeZone::Local) {
le TimeConverter converter;
/**
Queries the current time in the specified time zone.
@param [time_zone] The time zone to query. Defaults to local.
@return The current time in the specified time zone.
*/
fn now(TimeZone time_zone = TimeZone::Local) {
le TimeConverter converter;
if (time_zone == TimeZone::Local)
converter.value = builtin::std::time::to_local(std::time::epoch());
else if (time_zone == TimeZone::UTC)
converter.value = builtin::std::time::to_utc(std::time::epoch());
else
converter.value = 0x00;
if (time_zone == TimeZone::Local)
converter.value = builtin::std::time::to_local(std::time::epoch());
else if (time_zone == TimeZone::UTC)
converter.value = builtin::std::time::to_utc(std::time::epoch());
else
converter.value = 0x00;
return converter.time;
};
return converter.time;
};
/**
Converts a value to a DOS date.
@param value The value to convert.
@return The DOS date.
*/
fn to_dos_date(u16 value) {
le impl::DOSDateConverter converter;
/**
Converts a value to a DOS date.
@param value The value to convert.
@return The DOS date.
*/
fn to_dos_date(u16 value) {
le impl::DOSDateConverter converter;
converter.value = value;
converter.value = value;
return converter.date;
};
return converter.date;
};
/**
Converts a value to a DOS time.
@param value The value to convert.
@return The DOS time.
*/
fn to_dos_time(u16 value) {
le impl::DOSTimeConverter converter;
/**
Converts a value to a DOS time.
@param value The value to convert.
@return The DOS time.
*/
fn to_dos_time(u16 value) {
le impl::DOSTimeConverter converter;
converter.value = value;
converter.value = value;
return converter.time;
};
return converter.time;
};
/**
Converts a FILETIME to unix time.
@param value The value to convert.
@return Timestamp formatted as unix time.
*/
fn filetime_to_unix(u64 value) {
return value / 10000000 - 11644473600;
};
/**
Converts a FILETIME to unix time.
@param value The value to convert.
@return Timestamp formatted as unix time.
*/
fn filetime_to_unix(u64 value) {
return value / 10000000 - 11644473600;
};
/**
Formats a time according to the specified format string.
@param time The time to format.
@param [format_string] The format string to use. Defaults to "%c".
@return The formatted time.
*/
fn format(Time time, str format_string = "%c") {
le TimeConverter converter;
converter.time = time;
/**
Formats a time according to the specified format string.
@param time The time to format.
@param [format_string] The format string to use. Defaults to "%c".
@return The formatted time.
*/
fn format(Time time, str format_string = "%c") {
le TimeConverter converter;
converter.time = time;
return builtin::std::time::format(format_string, converter.value);
};
return builtin::std::time::format(format_string, converter.value);
};
/**
Formats a DOS date according to the specified format string.
@param date The DOS date to format.
@param [format_string] The format string to use. Defaults to "{}/{}/{}".
@return The formatted DOS date.
*/
fn format_dos_date(DOSDate date, str format_string = "{}/{}/{}") {
return std::format(format_string, date.day, date.month, date.year + 1980);
};
/**
Formats a DOS date according to the specified format string.
@param date The DOS date to format.
@param [format_string] The format string to use. Defaults to "{}/{}/{}".
@return The formatted DOS date.
*/
fn format_dos_date(DOSDate date, str format_string = "{}/{}/{}") {
return std::format(format_string, date.day, date.month, date.year + 1980);
};
/**
Formats a DOS time according to the specified format string.
@param time The DOS time to format.
@param [format_string] The format string to use. Defaults to "{:02}:{:02}:{:02}".
@return The formatted DOS time.
*/
fn format_dos_time(DOSTime time, str format_string = "{:02}:{:02}:{:02}") {
return std::format(format_string, time.hours, time.minutes, time.seconds * 2);
};
/**
Formats a DOS time according to the specified format string.
@param time The DOS time to format.
@param [format_string] The format string to use. Defaults to "{:02}:{:02}:{:02}".
@return The formatted DOS time.
*/
fn format_dos_time(DOSTime time, str format_string = "{:02}:{:02}:{:02}") {
return std::format(format_string, time.hours, time.minutes, time.seconds * 2);
};
}

View File

@@ -34,7 +34,7 @@ namespace auto type {
*/
using Bin<T> = T [[format("type::impl::format_bin")]];
namespace impl {
namespace impl {
fn format_number(auto value, str fmt) {
bool negative = value < 0;
@@ -50,6 +50,6 @@ namespace auto type {
fn format_dec(auto value) { return type::impl::format_number(value, "{}"); };
fn format_bin(auto value) { return type::impl::format_number(value, "0b{:08b}"); };
}
}
}

View File

@@ -1,6 +1,7 @@
#pragma once
import std.io;
import std.core;
/*!
Types to display single bytes using various different representations
@@ -20,7 +21,7 @@ namespace auto type {
bit5 : 1;
bit6 : 1;
bit7 : 1;
} [[format("type::impl::format_bits"), right_to_left]];
} [[format("type::impl::format_bits"), bitfield_order(std::core::BitfieldOrder::LeastToMostSignificant, 8)]];
/**
Type visualizing the value of the two nibbles
@@ -40,7 +41,7 @@ namespace auto type {
} [[format("type::impl::format_byte"), single_color]];
namespace impl {
namespace impl {
fn format_byte(Byte byte) {
return std::format("0x{0:02X} (0b{1:08b}) LSB:{2}, MSB:{3}",
@@ -66,6 +67,6 @@ namespace auto type {
return std::format("{{ {0:0X}, {1:0X} }}", nibbles.high, nibbles.low);
};
}
}
}

View File

@@ -22,7 +22,7 @@ namespace auto type {
float floatValue;
};
fn format_float16(float16 value) {
fn format_float16(float16 value) {
u32 sign = value >> 15;
u32 exponent = (value >> 10) & 0x1F;
u32 mantissa = value & 0x3FF;

42
includes/type/fmt.pat Normal file
View File

@@ -0,0 +1,42 @@
#pragma once
import std.io;
/*!
Type that allows specifying its format value using a format string.
## Usage
The following code reads a u32 from the data and formats it as an upper case hexadecimal value with
a minimum of 8 digits which is prefixed by 0x.
The format string is the same as passed to `std::format()` and follows the libfmt specification.
```rust
type::Formatted<u32, "0x{:08X}"> hex_formatted_integer @ 0x00;
```
*/
namespace auto type {
/**
Arbitrarily formatted type
@tparam T Type to format
@tparam FormatString libfmt format string to format the value
*/
struct Formatted<T, auto FormatString> {
T value;
} [[sealed, format("type::impl::format_formatted"), transform("type::impl::transform_formatted")]];
namespace impl {
fn format_formatted(ref auto formatted) {
return std::format(std::format("{{0:{}}}", formatted.FormatString), formatted.value);
};
fn transform_formatted(ref auto formatted) {
return formatted.value;
};
}
}

View File

@@ -3,15 +3,15 @@
import std.io;
/*!
Types to deal with UUIDs (Universally Unique Identifiers) / GUIDs (Globally Unique Identifiers) as described in RFC 4122
Types to deal with UUIDs (Universally Unique Identifiers) / GUIDs (Globally Unique Identifiers) as described in RFC 4122
*/
namespace auto type {
/**
Type representing a GUID value
*/
struct GUID {
/**
Type representing a GUID value
*/
struct GUID {
u32 time_low;
u16 time_mid;
u16 time_high_and_version;
@@ -20,31 +20,31 @@ namespace auto type {
u8 node[6];
} [[sealed, format("type::impl::format_guid")]];
/**
Alias name for GUID
*/
using UUID = GUID;
/**
Alias name for GUID
*/
using UUID = GUID;
namespace impl {
namespace impl {
fn format_guid(GUID guid) {
bool valid = ((le u16(guid.time_high_and_version) >> 12) <= 5) && (((guid.clock_seq_and_reserved >> 4) >= 8) || ((guid.clock_seq_and_reserved >> 4) == 0));
return std::format("{}{{{:08X}-{:04X}-{:04X}-{:02X}{:02X}-{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}}}",
valid ? "" : "Invalid ",
le u32(guid.time_low),
le u16(guid.time_mid),
le u16(guid.time_high_and_version),
guid.clock_seq_and_reserved,
guid.clock_seq_low,
guid.node[0],
guid.node[1],
guid.node[2],
guid.node[3],
guid.node[4],
guid.node[5]);
};
fn format_guid(GUID guid) {
bool valid = ((le u16(guid.time_high_and_version) >> 12) <= 5) && (((guid.clock_seq_and_reserved >> 4) >= 8) || ((guid.clock_seq_and_reserved >> 4) == 0));
return std::format("{}{{{:08X}-{:04X}-{:04X}-{:02X}{:02X}-{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}}}",
valid ? "" : "Invalid ",
le u32(guid.time_low),
le u16(guid.time_mid),
le u16(guid.time_high_and_version),
guid.clock_seq_and_reserved,
guid.clock_seq_low,
guid.node[0],
guid.node[1],
guid.node[2],
guid.node[3],
guid.node[4],
guid.node[5]);
};
}
}
}

View File

@@ -4,69 +4,69 @@ import std.io;
import std.mem;
/*!
Types used to decode Little Endian Base 128 numbers used to store large numbers as space efficiently as possible
Types used to decode Little Endian Base 128 numbers used to store large numbers as space efficiently as possible
*/
namespace auto type {
/**
Base LEB128 type. Use `uLEB128` and `sLEB128` instead.
*/
struct LEB128Base {
u8 array[while($ == addressof(this) || std::mem::read_unsigned($-1, 1) & 0x80 != 0)] [[hidden]];
} [[sealed]];
/**
Base LEB128 type. Use `uLEB128` and `sLEB128` instead.
*/
struct LEB128Base {
u8 array[while($ == addressof(this) || std::mem::read_unsigned($-1, 1) & 0x80 != 0)] [[hidden]];
} [[sealed]];
/**
A unsigned variant of a LEB128 number
*/
using uLEB128 = LEB128Base [[format("type::impl::format_uleb128"), transform("type::impl::transform_uleb128")]];
/**
A unsigned variant of a LEB128 number
*/
using uLEB128 = LEB128Base [[format("type::impl::format_uleb128"), transform("type::impl::transform_uleb128")]];
/**
A signed variant of a LEB128 number
*/
using sLEB128 = LEB128Base [[format("type::impl::format_sleb128"), transform("type::impl::transform_sleb128")]];
/**
A signed variant of a LEB128 number
*/
using sLEB128 = LEB128Base [[format("type::impl::format_sleb128"), transform("type::impl::transform_sleb128")]];
/**
Legacy alias for uLEB128
*/
using LEB128 = uLEB128;
namespace impl {
fn transform_uleb128_array(ref auto array) {
u128 res = array[0] & 0x7f;
for(u8 i = 1, array[i-1] & 0x80 != 0, i+=1) {
res |= u128(array[i] & 0x7f) << 7 * i;
}
return res;
};
/**
Legacy alias for uLEB128
*/
using LEB128 = uLEB128;
namespace impl {
fn transform_uleb128_array(ref auto array) {
u128 res = array[0] & 0x7f;
for(u8 i = 1, array[i-1] & 0x80 != 0, i+=1) {
res |= u128(array[i] & 0x7f) << 7 * i;
}
return res;
};
fn transform_sleb128_array(ref auto array) {
s128 res = type::impl::transform_uleb128_array(array);
if (res & 0x40 != 0) {
res |= ~0 << (sizeof(array) / sizeof(u8)) * 7;
}
return res;
};
fn format_uleb128(ref auto leb128) {
u128 res = type::impl::transform_uleb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_uleb128(ref auto leb128) {
return type::impl::transform_uleb128_array(leb128.array);
};
fn transform_sleb128_array(ref auto array) {
s128 res = type::impl::transform_uleb128_array(array);
if (res & 0x40 != 0) {
res |= ~0 << (sizeof(array) / sizeof(u8)) * 7;
}
return res;
};
fn format_uleb128(ref auto leb128) {
u128 res = type::impl::transform_uleb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_uleb128(ref auto leb128) {
return type::impl::transform_uleb128_array(leb128.array);
};
fn format_sleb128(ref auto leb128) {
s128 res = type::impl::transform_sleb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_sleb128(ref auto leb128) {
return type::impl::transform_sleb128_array(leb128.array);
};
fn format_sleb128(ref auto leb128) {
s128 res = type::impl::transform_sleb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_sleb128(ref auto leb128) {
return type::impl::transform_sleb128_array(leb128.array);
};
}
}
}

View File

@@ -3,30 +3,30 @@
import std.io;
/*!
Types used to decode MAC Addresses
Types used to decode MAC Addresses
*/
namespace auto type {
/**
A MAC Address as used in the Internet Protocol
*/
struct MACAddress {
u8 bytes[6];
} [[sealed, format("type::impl::format_mac_address")]];
/**
A MAC Address as used in the Internet Protocol
*/
struct MACAddress {
u8 bytes[6];
} [[sealed, format("type::impl::format_mac_address")]];
namespace impl {
namespace impl {
fn format_mac_address(MACAddress address) {
return std::format("{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}",
address.bytes[0],
address.bytes[1],
address.bytes[2],
address.bytes[3],
address.bytes[4],
address.bytes[5]);
};
return std::format("{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}",
address.bytes[0],
address.bytes[1],
address.bytes[2],
address.bytes[3],
address.bytes[4],
address.bytes[5]);
};
}
}
}

View File

@@ -1,53 +1,53 @@
import std.mem;
/*!
Types dealing with various kinds of resource paths
Types dealing with various kinds of resource paths
*/
namespace auto type {
/**
Type representing a single path segment. Use the `Path` type instead of using this on its own
@tparam Delimeter The delimeter sequence used to separate two path segments
*/
struct PathSegment<auto Delimeter> {
char string[while(std::mem::read_string($, std::string::length(Delimeter)) != Delimeter && std::mem::read_unsigned($, 1) != 0x00)];
char separator [[hidden]];
if (separator == 0x00) {
$ -= 1;
break;
}
} [[sealed, format("type::impl::format_path_segment")]];
/**
A generic type representing a path with an arbitrary delimeter
@tparam Delimeter The delimeter sequence used to separate two path segments
*/
struct Path<auto Delimeter> {
PathSegment<Delimeter> segments[while(true)];
} [[format("type::impl::format_path")]];
/**
A type representing a Unix path using a '/' forwardslash as delimeter
*/
using UnixPath = Path<"/">;
/**
Type representing a single path segment. Use the `Path` type instead of using this on its own
@tparam Delimeter The delimeter sequence used to separate two path segments
*/
struct PathSegment<auto Delimeter> {
char string[while(std::mem::read_string($, std::string::length(Delimeter)) != Delimeter && std::mem::read_unsigned($, 1) != 0x00)];
char separator [[hidden]];
if (separator == 0x00) {
$ -= 1;
break;
}
} [[sealed, format("type::impl::format_path_segment")]];
/**
A generic type representing a path with an arbitrary delimeter
@tparam Delimeter The delimeter sequence used to separate two path segments
*/
struct Path<auto Delimeter> {
PathSegment<Delimeter> segments[while(true)];
} [[format("type::impl::format_path")]];
/**
A type representing a Unix path using a '/' forwardslash as delimeter
*/
using UnixPath = Path<"/">;
/**
A type representing a DOS path using a '\' backslash as delimeter
*/
using DOSPath = Path<"\\">;
namespace impl {
fn format_path_segment(ref auto segment) {
return segment.string;
};
fn format_path(ref auto path) {
return std::mem::read_string($, sizeof(path));
};
}
/**
A type representing a DOS path using a '\\' backslash as delimeter
*/
using DOSPath = Path<"\\">;
namespace impl {
fn format_path_segment(ref auto segment) {
return segment.string;
};
fn format_path(ref auto path) {
return std::mem::read_string($, sizeof(path));
};
}
}

View File

@@ -1,4 +1,5 @@
#pragma description 7z File Format
#pragma MIME application/x-7z-compressed
import std.io;
import std.mem;

View File

@@ -1,12 +1,15 @@
#pragma author AdventureT
#pragma description Crash Bandicoot - Back in Time (fan game) User created level format
// Supports all versions till 0.94c, newer versions might be compatible!
#pragma history
#pragma 0.3 2024-05-15 Added support for version 0.95
#pragma 0.2 2023-10-29 Added support for version 0.94c
#pragma 0.1 2023-04-25 Initial support
// Supports all versions till 0.95, newer versions might be compatible!
import type.magic;
import std.string;
import std.array;
struct Header {
type::Magic<"CRASHLVL"> magic;
u8 version;
@@ -17,8 +20,6 @@ struct Header {
std::string::SizedString<u8> author;
};
Header header @ 0x0;
// Background Music
enum BGM : u32 {
None,
@@ -37,6 +38,56 @@ enum BGM : u32 {
};
enum BGMV2 : u32 {
None,
N_TropyBGM,
CrashCreatorBGM,
MainMenuBGM,
WarpRoomBGM,
Jungle01BGM,
SnowBGM,
RiverBGM,
FutureBGM,
LabBGM,
SewerBGM,
EgyptBGM,
NBrioBGM,
AdventureBGM,
SpyBGM,
ChaseBGM,
TrialsBGM,
SpaceBGM,
Jungle02BGM,
RipperBGM,
TheGreatWallBGM,
RoadToSomewhereBGM,
LavaKoalaBGM,
CortexBGM,
CyberCortexBGM,
ArabicBGM,
N_Tropy2BGM,
JazzBGM,
Space2BGM,
TawnaBonusBGM,
CortexPowerBGM,
ArabicBonusBGM,
EgyptBonusBGM,
FutureBonusBGM,
LostCityBGM,
PolarBGM,
RiverBonusBGM,
RuinsBonusBGM,
SewerBonusBGM,
SnowBonusBGM,
RoadToRuinBGM,
NGinBGM,
Arabia01BGM,
Arabia02BGM,
BashBGM,
Cortex02BGM
};
// v0.95
enum BGMV3 : BGMV2 {
None,
N_TropyBGM,
CrashCreatorBGM,
@@ -83,12 +134,19 @@ enum BGMV2 : u32 {
Arabia02BGM,
BashBGM,
Cortex02BGM,
MedievalBGM,
PreHistoricBGM,
UnderWaterBGM,
BrioRevisitedBGM,
EgyptChaseBGM,
RuinsLoopBGM,
DingoSynthBGM
};
enum Type : u32 {
Unset,
Flashback,
Trial,
Trial
};
enum TypeV2 : u32 {
@@ -118,13 +176,31 @@ enum SkyboxV2 : u32 {
Black
};
// 0.95
enum SkyboxV3 : u32 {
Default = 1,
Briolab,
Fort,
Moon,
Toxic,
AboutRight,
Crash1Island,
Arabia,
RoadToRuin,
MotorcycleDay,
MotorcycleNoon,
MotorcycleMoon,
MotorcycleNight,
Black
};
enum Scenery : u32 {
None,
FutureTense,
Forest,
Waterfall,
Snow,
Fortress,
Fortress
};
enum SceneryV2 : u32 {
@@ -165,44 +241,102 @@ enum SceneryV4 : u32 {
Pipes
};
// 0.95
enum SceneryV5 : u32 {
None,
FutureTense,
Forest,
Waterfall,
Snow,
Fortress,
None2,
Lava,
TheGreatGate,
Mountain,
KoalaKong,
SunsetVista,
HangemHigh,
Sphynxinator,
Tunnel,
Pipes,
Medieval,
FutureCity,
TinyArena,
HeavyMachinery,
CrystalCave,
MedievalWithHouses,
CortexBonusChamber
};
enum Weather : u32 {
Default,
Snow,
Rain
};
// 0.95
enum WeatherV2 : u32 {
Default,
Snow,
Rain,
Night,
UnderWater
};
struct Options {
if (header.version > 1)
// Type
if (header.version > 1) {
TypeV2 type;
else
Type type;
if (header.version > 1)
SkyboxV2 skybox;
else
Skybox skybox;
if (header.version == 1)
Scenery scenery;
else if (header.version > 1 && header.version < 4)
SceneryV2 scenery;
else
SceneryV4 scenery;
if (header.version > 2)
{
Weather weather;
}
if (header.version > 1)
else {
Type type;
}
// Skybox
if (header.version > 1) {
if (header.gameVersion == "0.95") {
SkyboxV3 skybox;
}
else {
SkyboxV2 skybox;
}
}
else {
Skybox skybox;
}
// Scenery
if (header.version == 1) {
Scenery scenery;
}
else if (header.version > 1 && header.version < 4) {
SceneryV2 scenery;
}
else {
if (header.gameVersion == "0.95") {
SceneryV5 skybox;
}
else {
SceneryV4 skybox;
}
}
// Weather
if (header.version > 2) {
if (header.gameVersion == "0.95") {
WeatherV2 weather;
}
else {
Weather weather;
}
}
// Background music
if (header.version > 1) {
BGMV2 bgm;
else
}
else {
BGM bgm;
if (type == Type::Trial)
{
}
// Time Trial
if (type == Type::Trial) {
u32 timeTrialTicksBronze;
u32 timeTrialTicksSilver;
u32 timeTrialTicksGold;
@@ -210,22 +344,17 @@ struct Options {
};
struct Object {
std::string::SizedString<u8> objName;
if (header.version > 1)
{
if (header.version > 1) {
u16 x;
u16 y;
bool hasMetafields;
if (hasMetafields)
{
if (hasMetafields) {
u16 numOfMetafields;
u8 metafields[numOfMetafields];
}
}
else
{
}
else {
u32 x;
u32 y;
}
@@ -236,9 +365,6 @@ struct Objects{
std::Array<Object, objCount> objArray;
};
Header header @ $;
Options options @ $;
Objects objects @ $;
Objects objects @ $;

66
patterns/ani.hexpat Normal file
View File

@@ -0,0 +1,66 @@
#pragma description Windows animated cursor
#pragma MIME application/x-navi-animation
#pragma endian little
import std.io;
import std.mem;
bitfield HeaderFlags {
icon : 1;
sequence : 1;
padding: 30;
};
struct anih {
u32 struct_size;
u32 stored_frames;
u32 animation_steps;
u32 w;
u32 h;
u32 bits;
u32 planes;
u32 default_jiffies;
HeaderFlags flags;
};
struct rate {
u32 jiffies[parent.size / 4];
};
struct seq {
u32 index[parent.size / 4];
};
struct RiffChunk {
char signature[4];
u32 size;
if (signature == "RIFF" || signature == "LIST") {
char type[4];
RiffChunk chunks[while($ - addressof(type) < size)];
} else if (signature[0] == 'I' && parent.type == "INFO") {
char info[size];
} else if (signature == "anih") {
anih anih [[inline]];
} else if (signature == "rate") {
rate rate [[inline]];
} else if (signature == "seq ") {
seq seq [[inline]];
} else {
std::mem::Bytes<size> data;
}
padding[size % 2];
} [[format_read("read_chunk")]];
fn read_chunk(RiffChunk chunk) {
if (chunk.signature == "RIFF" || chunk.signature == "LIST") {
return std::format("{}<{}> ({})", chunk.signature, chunk.type, chunk.size);
} else {
str ret = std::format("{} ({})", chunk.signature, chunk.size);
try {
ret += std::format(" \"{}\"", chunk.info);
} catch {}
return ret;
}
};
RiffChunk riff @ 0;

View File

@@ -1,3 +1,5 @@
#pragma description TODO
import std.math;
import std.core;
import type.magic;

View File

@@ -167,16 +167,26 @@ bitfield Caps2Flags {
};
bitfield PixelFormatFlags {
alphaPixels : 1;
alpha : 1;
fourCC : 1;
padding : 3;
rgb : 1; // 0x40
padding : 2;
yuv : 1; // 0x200
padding : 3;
luminance : 1; // 0x20000
padding : 17;
alphaPixels : 1; // DDPF_ALPHAPIXELS
alpha : 1; // DDPF_ALPHA
fourCC : 1; // DDPF_FOURCC
paletteIndexed4 : 1; // DDPF_PALETTEINDEXED4
paletteIndexedTo8 : 1; // DDPF_PALETTEINDEXEDTO8
paletteIndexed8 : 1; // DDPF_PALETTEINDEXED8
rgb : 1; // DDPF_RGB
compressed : 1; // DDPF_COMPRESSED
rgbToYuv : 1; // DDPF_RGBTOYUV
yuv : 1; // DDPF_YUV
zBuffer : 1; // DDPF_ZBUFFER
paletteIndexed1 : 1; // DDPF_PALETTEINDEXED1
paletteIndexed2 : 1; // DDPF_PALETTEINDEXED2
zPixels : 1; // DDPF_ZPIXELS
stencilBuffer : 1; // DDPF_STENCILBUFFER
alphaResult : 1; // DDPF_ALPHARESULT
luminance : 1; // DDPF_LUMINANCE
bumpLuminance : 1; // DDPF_BUMPLUMINANCE
bumpDudv : 1; // DDPF_BUMPDUDV
padding : 13; // Padding bits to complete 32-bit structure
};
enum DX10ResourceDimension : u32 {

117
patterns/dted.hexpat Normal file
View File

@@ -0,0 +1,117 @@
#pragma description Digital Terrain Elevation Data
#pragma endian big
import std.core;
import std.io;
import std.mem;
import std.string;
enum Magic:u24 {
UHL = 0x55484C,
DSI = 0x445349,
ACC = 0x414343,
};
struct UHL {
Magic magic;
char one;
char lon[8];
char lat[8];
char lon_data_interval[4];
char lat_data_interval[4];
char accuracy[4];
char security_code[3];
char uniq_ref[12];
char lon_lines[4];
char lat_points[4];
char multi_accuracy;
char reserved[24];
};
struct DSI {
Magic magic;
char classification;
char stuff1[29];
char stuff2[26];
char product_level[5];
char uniq_ref[15];
char reserved[8];
char data_edition[2];
char match_version;
char maint_date[4];
char match_date[4];
char main_desc_code[4];
char producer_code[8];
char reserved2[16];
char product_spec[9];
char numbers[2];
char product_spec_date[4];
char vertical_datum[3];
char horizontal_datum[5];
char digitizing_system[10];
char compilation_date[4];
char reserved3[22];
char lat_origin[9];
char lon_origin[10];
char lat_sw_corner[7];
char lon_sw_corner[8];
char lat_nw_corner[7];
char lon_nw_corner[8];
char lat_ne_corner[7];
char lon_ne_corner[8];
char lat_se_corner[7];
char lon_se_corner[8];
char clockwise_orientation[9];
char lat_interval[4];
char lon_interval[4];
char lat_lines[4];
char lon_lines[4];
char partial_cell[2];
char reserved4[101];
char reserved5[100];
char reserved6[156];
};
struct ACCSub {
char abs_vertical_accuracy[4];
char abs_horizontal_accuracy[4];
char rel_vertical_accuracy[4];
char rel_horizontal_accuracy[4];
char num_coords[2];
char pairs[19*14];
};
struct ACC {
Magic magic;
char abs_horizontal_accuracy[4];
char abs_vertical_accuracy[4];
char rel_horizontal_accuracy[4];
char rel_vertical_accuracy[4];
char reserved1[4];
char reserved2[1];
char reserved3[31];
char multi_accuracy_outline[2]; // determines sub regions
ACCSub subs[9];
char reserved4[18];
char reserved5[69];
};
struct DataRecords {
char magic;
s24 data_block_count;
s16 lon_count;
s16 lat_count;
s16 elevation[std::string::parse_int(parent.dsi.lat_lines, 10)];
u32 checksum;
};
struct DTED {
UHL uhl;
DSI dsi;
ACC acc;
DataRecords records[std::string::parse_int(this.dsi.lon_lines, 10)];
};
DTED dted @ 0x00;

View File

@@ -498,12 +498,12 @@ bitfield SHF {
bitfield ELF32_R_INFO {
SYM : 8;
TYPE : 8;
} [[left_to_right]];
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 16)]];
bitfield ELF64_R_INFO {
SYM : 32;
TYPE : 32;
} [[left_to_right]];
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 64)]];
bitfield PF {
X : 1;

View File

@@ -52,7 +52,7 @@ namespace fat32 {
lastLogical : 1;
padding : 1;
number : 5;
} [[left_to_right]];
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 8)]];
enum EntryStatus : u8 {
Regular = 0x00,
@@ -73,7 +73,7 @@ namespace fat32 {
subdirectory : 1;
archive : 1;
padding : 2;
} [[right_to_left]];
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 8)]];
struct DirEntry {
char fileName[8];

215
patterns/gguf.hexpat Normal file
View File

@@ -0,0 +1,215 @@
// https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
// https://github.com/openxla/iree/blob/main/runtime/src/iree/io/formats/gguf/gguf_parser.c
#pragma description GGUF v3 File Format Patter
#pragma authors @leonjza, jessie @ imhex discord
#pragma pattern_limit 300000
enum ggml_type: u32 {
GGML_TYPE_F32 = 0,
GGML_TYPE_F16 = 1,
GGML_TYPE_Q4_0 = 2,
GGML_TYPE_Q4_1 = 3,
// GGML_TYPE_Q4_2 = 4, support has been removed
// GGML_TYPE_Q4_3 = 5, support has been removed
GGML_TYPE_Q5_0 = 6,
GGML_TYPE_Q5_1 = 7,
GGML_TYPE_Q8_0 = 8,
GGML_TYPE_Q8_1 = 9,
GGML_TYPE_Q2_K = 10,
GGML_TYPE_Q3_K = 11,
GGML_TYPE_Q4_K = 12,
GGML_TYPE_Q5_K = 13,
GGML_TYPE_Q6_K = 14,
GGML_TYPE_Q8_K = 15,
GGML_TYPE_IQ2_XXS = 16,
GGML_TYPE_IQ2_XS = 17,
GGML_TYPE_IQ3_XXS = 18,
GGML_TYPE_IQ1_S = 19,
GGML_TYPE_IQ4_NL = 20,
GGML_TYPE_IQ3_S = 21,
GGML_TYPE_IQ2_S = 22,
GGML_TYPE_IQ4_XS = 23,
GGML_TYPE_I8 = 24,
GGML_TYPE_I16 = 25,
GGML_TYPE_I32 = 26,
GGML_TYPE_I64 = 27,
GGML_TYPE_F64 = 28,
GGML_TYPE_IQ1_M = 29,
GGML_TYPE_COUNT,
};
enum gguf_metadata_value_type: u32 {
// The value is a 8-bit unsigned integer.
GGUF_METADATA_VALUE_TYPE_UINT8 = 0,
// The value is a 8-bit signed integer.
GGUF_METADATA_VALUE_TYPE_INT8 = 1,
// The value is a 16-bit unsigned little-endian integer.
GGUF_METADATA_VALUE_TYPE_UINT16 = 2,
// The value is a 16-bit signed little-endian integer.
GGUF_METADATA_VALUE_TYPE_INT16 = 3,
// The value is a 32-bit unsigned little-endian integer.
GGUF_METADATA_VALUE_TYPE_UINT32 = 4,
// The value is a 32-bit signed little-endian integer.
GGUF_METADATA_VALUE_TYPE_INT32 = 5,
// The value is a 32-bit IEEE754 floating point number.
GGUF_METADATA_VALUE_TYPE_FLOAT32 = 6,
// The value is a boolean.
// 1-byte value where 0 is false and 1 is true.
// Anything else is invalid, and should be treated as either the model being invalid or the reader being buggy.
GGUF_METADATA_VALUE_TYPE_BOOL = 7,
// The value is a UTF-8 non-null-terminated string, with length prepended.
GGUF_METADATA_VALUE_TYPE_STRING = 8,
// The value is an array of other values, with the length and type prepended.
///
// Arrays can be nested, and the length of the array is the number of elements in the array, not the number of bytes.
GGUF_METADATA_VALUE_TYPE_ARRAY = 9,
// The value is a 64-bit unsigned little-endian integer.
GGUF_METADATA_VALUE_TYPE_UINT64 = 10,
// The value is a 64-bit signed little-endian integer.
GGUF_METADATA_VALUE_TYPE_INT64 = 11,
// The value is a 64-bit IEEE754 floating point number.
GGUF_METADATA_VALUE_TYPE_FLOAT64 = 12,
};
// A string in GGUF.
struct gguf_string_t {
// The length of the string, in bytes.
u64 len;
// The string as a UTF-8 non-null-terminated string.
char string[len];
};
struct gguf_metadata_value_t {
gguf_metadata_value_type type;
u64 length;
match(type) {
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT8): u8 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT8): s8 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT16): u16 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT16): s16 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT32): u32 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT32): s32 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT32): float value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_BOOL): bool value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_STRING): gguf_string_t value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT64): u64 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT64): double value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_ARRAY): double value[length];
}
};
struct gguf_metadata_value {
gguf_metadata_value_type type;
match(type) {
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT8): u8 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT8): s8 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT16): u16 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT16): s16 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT32): u32 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT32): s32 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT32): float value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_BOOL): bool value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_STRING): gguf_string_t value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT64): u64 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT64): double value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_ARRAY): gguf_metadata_value_t value;
}
};
struct gguf_metadata_kv_t {
// The key of the metadata. It is a standard GGUF string, with the following caveats:
// - It must be a valid ASCII string.
// - It must be a hierarchical key, where each segment is `lower_snake_case` and separated by a `.`.
// - It must be at most 2^16-1/65535 bytes long.
// Any keys that do not follow these rules are invalid.
gguf_string_t key;
// The type of the value.
// Must be one of the `gguf_metadata_value_type` values.
// gguf_metadata_value_type value_type;
// The value.
gguf_metadata_value value;
};
struct gguf_header_t {
// Magic number to announce that this is a GGUF file.
// Must be `GGUF` at the byte level: `0x47` `0x47` `0x55` `0x46`.
// Your executor might do little-endian byte order, so it might be
// check for 0x46554747 and letting the endianness cancel out.
// Consider being *very* explicit about the byte order here.
u32 magic;
// The version of the format implemented.
// Must be `3` for version described in this spec, which introduces big-endian support.
//
// This version should only be increased for structural changes to the format.
// Changes that do not affect the structure of the file should instead update the metadata
// to signify the change.
u32 version;
// The number of tensors in the file.
// This is explicit, instead of being included in the metadata, to ensure it is always present
// for loading the tensors.
u64 tensor_count;
// The number of metadata key-value pairs.
u64 metadata_kv_count;
// The metadata key-value pairs.
gguf_metadata_kv_t metadata_kv[metadata_kv_count];
};
struct gguf_tensor_info_t {
// The name of the tensor. It is a standard GGUF string, with the caveat that
// it must be at most 64 bytes long.
gguf_string_t name;
// The number of dimensions in the tensor.
// Currently at most 4, but this may change in the future.
u32 n_dimensions;
// The dimensions of the tensor.
u64 dimensions[n_dimensions];
// The type of the tensor.
ggml_type type;
// The offset of the tensor's data in this file in bytes.
//
// This offset is relative to `tensor_data`, not to the start
// of the file, to make it easier for writers to write the file.
// Readers should consider exposing this offset relative to the
// file to make it easier to read the data.
//
// Must be a multiple of `ALIGNMENT`. That is, `align_offset(offset) == offset`.
u64 offset;
};
struct gguf_file_t {
// The header of the file.
gguf_header_t header;
// Tensor infos, which can be used to locate the tensor data.
gguf_tensor_info_t tensor_infos[header.tensor_count];
// Padding to the nearest multiple of `ALIGNMENT`.
//
// That is, if `sizeof(header) + sizeof(tensor_infos)` is not a multiple of `ALIGNMENT`,
// this padding is added to make it so.
//
// This can be calculated as `align_offset(position) - position`, where `position` is
// the position of the end of `tensor_infos` (i.e. `sizeof(header) + sizeof(tensor_infos)`).
u8 _padding[];
// Tensor data.
//
// This is arbitrary binary data corresponding to the weights of the model. This data should be close
// or identical to the data in the original model file, but may be different due to quantization or
// other optimizations for inference. Any such deviations should be recorded in the metadata or as
// part of the architecture definition.
//
// Each tensor's data must be stored within this array, and located through its `tensor_infos` entry.
// The offset of each tensor's data must be a multiple of `ALIGNMENT`, and the space between tensors
// should be padded to `ALIGNMENT` bytes.
u8 tensor_data[];
};
gguf_file_t GGUF @ 0x00;

View File

@@ -22,6 +22,8 @@
* SOFTWARE.
*/
#pragma description TODO
import std.mem;
import std.io;

View File

@@ -1,536 +0,0 @@
import std.mem;
import std.io;
import std.string;
struct Header {
u32 magic;
u32 version;
u64 RootStructAltGUID;
u64 AssetChecksum;
u32 DependencyCount;
s32 DataBlockCount;
s32 TagStructCount;
s32 DataReferenceCount;
s32 TagReferenceCount;
u32 StringTableSize;
u32 ZoneSetDataSize;
u32 Unk_0x34;
u32 HeaderSize;
u32 DataSize;
u32 ResourceDataSize;
u32 ActualResoureDataSize;
u8 HeaderAlignment;
u8 TagDataAlignment;
u8 ResourceDataAlignment;
u8 ActualResourceDataAlignment;
s32 IsResource;
};
enum curve_mode : u8 {
choose_best = 0x00,
force_FAST = 0x01,
force_PRETTY = 0x02,
};
enum BitmapForcedFormat : u16 {
Use_Default = 0x00,
Best_Compressed_Color_Format = 0x01,
Best_Uncompressed_Color_Format = 0x02,
Best_Compressed_Bump_Format = 0x03,
Best_Uncompressed_Bump_Format = 0x04,
Best_Compressed_Monochrome_Format = 0x05,
Best_Uncompressed_Monochrome_Format = 0x06,
Best_Compressed_Monochrome_Format_Without_Alpha = 0x07,
unused2 = 0x08,
unused3 = 0x09,
unused4 = 0x0A,
unused5 = 0x0B,
unused6 = 0x0C,
Color_And_Alpha_Formats = 0x0D,
bc1_unorm_DXT1_Compressed_Color_Key_Alpha = 0x0E,
bc2_unorm_DXT3_Compressed_Color_4bit_Alpha = 0x0F,
bc3_unorm_DXT5_Compressed_Color_Compressed_8bit_Alpha = 0x10,
Color_24bit_Alpha_8bit = 0x11,
Monochrome_8bit_Alpha_8bit = 0x12,
Channel_Mask_3bit_Color_1bit_Alpha = 0x13,
Color_30bit_Alpha_2bit = 0x14,
Color_48bit_Alpha_16bit = 0x15,
HALF_Color_Alpha = 0x16,
FLOAT_Color_Alpha = 0x17,
r8_unorm_8bit_Intensity_RGBA = 0x18,
DEPRECATED_DXT3A_4bit_Intensity_ARGB = 0x19,
bc4_unorm_rrrr_DXT5A_Compressed_Intensity_ARGB = 0x1A,
Compressed_Monochrome_Alpha = 0x1B,
b4g4r4a4_12bit_Color_4bit_Alpha = 0x1C,
Color_Only_Formats = 0x1D,
Monochrome_8bit = 0x1E,
Compressed_Color_24bit = 0x1F,
Color_32bit_R11G11B10 = 0x20,
Monochrome_16bit = 0x21,
Red_Green_Only_16bit = 0x22,
Signed_RGBA_16bit = 0x23,
HALF_Red_Only = 0x24,
FLOAT_Red_Only = 0x25,
HALF_Red_Green_Only = 0x26,
FLOAT_Red_Green_Only = 0x27,
HALF_Monochrome = 0x28,
DEPRECATED_Compressed_Monochrome_4bit = 0x29,
Compressed_Interpolated_Monochrome = 0x2A,
bc6h_uf16_HDR_RGB_Unsigned_Half_Float = 0x2B,
bc6h_sf16_HDR_RGB_Signed_Half_Float = 0x2C,
bc7_unorm_High_Quality_bc3 = 0x2D,
Alpha_Only_Formats = 0x2E,
DEPRECATED_DXT3A_4bit_Alpha = 0x2F,
bc4_unorm_000r_DXT5A_8bit_Compressed_Alpha = 0x30,
Alpha_8bit = 0x31,
unused28 = 0x32,
unused29 = 0x33,
unused30 = 0x34,
Normal_Map_Formats = 0x35,
bc5_snorm_DXN_Compressed_Normals_Better = 0x36,
DEPRECATED_CTX1_Compressed_Normals_Smaller = 0x37,
Normals_16bit = 0x38,
Normals_32bit = 0x39,
Vector_8bit_4_Channel = 0x3A,
};
enum BitmapDownsampleFilter : u8 {
point_sampled = 0x01,
box_filter = 0x02,
blackman_filter = 0x03,
lanczos_filter = 0x04,
nuttall_filter = 0x05,
blackman_harris_filter = 0x06,
blackman_nuttall_filter = 0x07,
flat_top_filter = 0x08,
extreme_filter = 0x09,
min_filter = 0x0A
};
enum BitmapType : u8 {
_2D_Texture = 0x00,
_3D_Texture = 0x01,
CubeMap = 0x02,
Array = 0x03,
};
enum BitmapColorSpace : u8 {
Automatic = 0x00,
Rec709 = 0x01,
DCI_P3 = 0x02,
Rec2020 = 0x03,
};
enum SignedDistanceFieldMethod : u8 {
Neighbors_Multithreaded = 0x00,
Neighbors = 0x01,
Cached_Euclidean_Multithreaded = 0x02,
Cached_Euclidean = 0x03,
SSEDT = 0x04,
};
enum BitmapFormat : u16 {
a8_unorm = 0x00,
r8_unorm_rrr1 = 0x01,
r8_unorm_rrrr = 0x02,
r8g8_unorm_rrrg = 0x03,
unused1 = 0x04,
unused2 = 0x05,
b5g6r5_unorm = 0x06,
unused3 = 0x07,
b5g6r5a1_unorm = 0x08,
b4g4r4a4_unorm = 0x09,
b8g8r8x8_unorm = 0x0A,
b8g8r8a8_unorm = 0x0B,
unused4 = 0x0C,
DEPRECATED_dxt5_bias_alpha = 0x0D,
bc1_unorm_dxt1 = 0x0E,
bc2_unorm_dxt3 = 0x0F,
bc3_unorm_dxt5 = 0x10,
DEPRECATED_a4r4g4b4_font = 0x11,
unused7 = 0x12,
unused8 = 0x13,
DEPRECATED_SOFTWARE_rgbfp32 = 0x14,
unused9 = 0x15,
r8g8_snorm_v8u8 = 0x16,
DEPRECATED_g8b8 = 0x17,
r32g32b32a32_float_abgrfp32 = 0x18,
r16g16b16a16_float_abgrfp16 = 0x19,
r16_float_rrr1_16f_mono = 0x1A,
r16_float_r000_16f_red = 0x1B,
r8g8b8a8_snorm_q8w8v8u8 = 0x1C,
r10g10b10a2_unorm_a2r10g10b10 = 0x1D,
r16g16b16a16_unorm_a16b16g16r16 = 0x1E,
r16g16_snorm_v16u16 = 0x1F,
r16_unorm_rrr0_L16 = 0x20,
r16g16_unorm_r16g16 = 0x21,
r16g16b16a16_snorm_signedr16g16b16a16 = 0x22,
DEPRECATED_dxt3a = 0x23,
bc4_unorm_rrrr_dxt5a = 0x24,
bc4_snorm_rrrr = 0x25,
DEPRECATED_dxt3a_1111 = 0x26,
bc5_snorm_dxn = 0x27,
DEPRECATED_ctx1 = 0x28,
DEPRECATED_dxt3a_alpha_only = 0x29,
DEPRECATED_dxt3a_monochrome_only = 0x2A,
bc4_unorm_000r_dxt5a_alpha = 0x2B,
bc4_unorm_rrr1_dxt5a_mono = 0x2C,
bc5_unorm_rrrg_dxn_mono_alpha = 0x2D,
bc5_snorm_rrrg_dxn_mono_alpha_signed = 0x2E,
bc6h_uf16 = 0x2F,
bc6h_sf16 = 0x30,
bc7_unorm = 0x31,
d24_unorm_s8_uint_depth_24 = 0x32,
r11g11b10_float = 0x33,
r16g16_float = 0x34,
};
enum BitmapCurve : u8 {
unknown = 0x00,
xRGB = 0x01,
gamma_2 = 0x02,
linear = 0x03,
offset_log = 0x04,
sRGB = 0x05,
Rec709 = 0x06,
};
enum BitmapUsageFlag : u8 {
ignore_curve_override = 0x00,
do_not_allow_size_optimization = 0x01,
swap_axes = 0x02,
pre_filter_cubemaps = 0x03,
};
enum BitmapSlicer : u8 {
automatically_determine_slicer = 0x00,
no_slicing = 0x01,
color_plate_slicer = 0x02,
cube_map_slicer = 0x03,
color_grading_slicer = 0x04,
};
enum BitmapPacker : u8 {
no_packing = 0x00,
sprite_packing = 0x01,
sprite_packing_if_needed = 0x02,
_3d_pack = 0x03,
};
enum BitmapSmallestMip : u8 {
_1_pixel = 0x00,
_2_pixel = 0x01,
_4_pixel = 0x02,
_8_pixel = 0x03,
_16_pixel = 0x04,
_32_pixel = 0x05,
_64_pixel = 0x06,
_128_pixel = 0x07,
_256_pixel = 0x08,
_512_pixel = 0x09,
_1024_pixel = 0x0A,
};
enum Swizzle : u8 {
default = 0x00,
source_red_channel = 0x01,
source_green_channel = 0x02,
source_blue_channel = 0x03,
source_alpha_channel = 0x04,
set_to_1 = 0x05,
set_to_0 = 0x06,
set_to_0_5 = 0x07,
Random = 0x08,
one_minus_source_red_channel = 0x09,
one_minus_source_green_channel = 0x0A,
one_minus_source_blue_channel = 0x0B,
one_minus_source_alpha_channel = 0x0C,
negative_source_red_channel = 0x0D,
negative_source_green_channel = 0x0E,
negative_source_blue_channel = 0x0F,
negative_source_alpha_channel = 0x10,
};
bitfield BitmapDicerFlag {
convert_plate_color_key_to_alpha_channel : 1;
rotate_cube_map_to_match_directX_format : 1;
sprites_shrink_elements_to_smallest_non_zero_alpha_region : 1;
sprites_shrink_elements_to_smallest_non_zero_color_and_alpha_region : 1;
unsigned_signed_scale_and_bias : 1;
color_grading_sRGB_correction : 1;
color_grading_Rec709_correction : 1;
};
bitfield BitmapDownsampleFlag {
sprites : 1;
pre_multiply_alpha : 1;
post_divide_alpha : 1;
height_map : 1;
detail_map : 1;
_signed : 1;
illum_map : 1;
zbump_map : 1;
cubemap : 1;
calculate_spec_power : 1;
downsample_bumps_in_angular_space : 1;
standard_orientation_of_normals_in_angular_space_and_renormalize : 1;
generate_rgb_luminance_into_alpha_channel : 1;
};
bitfield BitmapPackerFlag {
shrink_sprite_texture_pages_tightly_to_content : 1;
};
bitfield BitmapDataResourceFlags {
Contains_3D_Texture : 1;
CPU_Gameplay_Required : 1;
Single_Mip_Texture : 1;
UI_Texture : 1;
};
bitfield BitmapGroupFlags {
bitmap_is_tiled : 1;
use_less_blurry_bump_map : 1;
dither_when_compressing : 1;
generate_random_sprites : 1;
ignore_alpha_channel : 1;
alpha_channel_stores_transparency : 1;
preserve_alpha_channel_in_mipmaps : 1;
only_use_on_demand : 1;
apply_max_resolution_after_slicing :1;
pre_filter_cubemaps : 1;
has_valid_min_and_max_values : 1;
SDF_Only_Store_Inside_values : 1;
SDF_Store_Direction : 1;
SDF_Store_Initial_Values_In_Alpha_Channel : 1;
Has_CUstom_Mipmaps : 1;
Validate_Has_Custom_Mipmaps : 1;
Is_Light_Probes : 1;
pad1 : 1;
pad2 : 1;
pad3 : 1;
pad4 : 1;
pad5 : 1;
pad6 : 1;
pad7 : 1;
pad8 : 1;
};
bitfield BitmapFlags {
power_of_two_dimension : 1;
compressed : 1;
swap_axes : 1;
mutable_at_runtime : 1;
pad1 : 1;
pad2 : 1;
pad3 : 1;
pad4 : 1;
pad5 : 1;
};
struct Data_Block {
u32 Size;
u16 Padding;
u16 Section;
u64 Offset;
};
struct Tag_Def_Structure {
s64 GUID_1;
s64 GUID_2;
u16 Type;
u16 Unk_0x12;
s32 TargetIndex;
s32 FieldBlock;
u32 FieldOffset;
};
struct Tag_Dependency {
char GroupTag[4] [[format("std::string::reverse")]];
u32 NameOffset;
u64 AssetID;
u32 GlobalID;
u32 Unk_0x14;
};
struct Data_Reference {
s32 ParentStructIndex;
s32 Unk_0x04;
s32 TargetIndex;
s32 FieldBlock;
u32 FieldOffset;
};
struct TagReference {
s64 GlobalHandle;
s32 ref_id_int;
s32 ref_id_sub;
s32 ref_id_center_int;
char TagGroupRev[4] [[format("std::string::reverse")]];
s32 local_handle;
};
struct Tag_Fixup_Reference {
s32 FieldBlock;
u32 FieldOffset;
u32 NameOffset;
s32 DependencyIndex;
TagReference tagreference @ header.HeaderSize + datablocks[FieldBlock].Offset + FieldOffset;
};
struct RealARgbColor {
float A;
float R;
float G;
float B;
};
struct RealRgbColor {
float R;
float G;
float B;
};
struct Bitmap_Resource_Handle {
s32 pixels[6]; // sum of these equals pixel count.
u32 hardware_format;
s8 tileMode;
s8 format;
BitmapDataResourceFlags bitmap_data_resource_flags;
s8 Alignment_Bits;
u8 highResMipCountAndFlags;
u8 mipCountPerArraySlice;
char generated_padff1b[6];
s64 runtime_data;
char streamingData[20];
s32 generated_padb266;
};
struct BitmapBlock {
u16 width;
u16 height;
u16 depth;
BitmapType Bitmap_Type;
u8 generated_paddca;
BitmapFormat bitmap_format;
BitmapFlags bitmap_flags;
s8 mipmap_count;
BitmapCurve bitmap_curve;
u16 generated_pad8fee;
float streaming_scale;
s16 source_width;
s16 source_height;
char bitmap_resource_handle_size[12];
s32 bitmap_resource_handle_count;
Bitmap_Resource_Handle bitmap_resource_handle[bitmap_resource_handle_count];
};
struct BitmapUsage {
s32 name;
s32 editor_group;
u32 source_gamma;
BitmapCurve bitmap_curve;
BitmapUsageFlag bitmap_usage_flag;
BitmapSlicer bitmap_slicer;
BitmapDicerFlag bitmap_dicer;
BitmapPacker bitmap_packer;
BitmapPackerFlag bitmap_packer_flags;
BitmapType bitmap_type;
s8 mipmap_limit;
BitmapSmallestMip bitmap_smallest_mip;
BitmapDownsampleFilter bitmap_downsample_filter;
s8 filter_radius_bias;
char generated_pad122e[1];
BitmapDownsampleFlag bitmap_downsample_flag;
char generated_pad5ee3[2];
RealRgbColor sprite_background_color;
Swizzle swizzle_red;
Swizzle swizzle_green;
Swizzle swizzle_blue;
Swizzle swizzle_alpha;
BitmapFormat bitmap_formats;
BitmapColorSpace source_color_space;
BitmapColorSpace target_color_space;
};
struct BitmapGroupSprite {
u32 bitmap_index;
char generated_pad2095[2];
s32 left;
s32 right;
s32 top;
s32 bottom;
s64 real_point_2D;
};
struct BitmapGroupSequence {
char name[32];
u32 first_bitmap_index;
u32 bitmap_count;
char sprites[16];
s32 sprite_count;
BitmapGroupSprite Sprites[sprite_count];
};
struct Bitm {
u32 Usage;
s32 UsageID;
s32 Package;
s32 texture_group;
BitmapGroupFlags flags;
u16 sprite_spacing;
u16 mip_sample_count;
float bump_map_height;
float blur_factor;
u32 blur;
u32 mipmap_blur;
curve_mode curvemode;
s8 max_mipmap_level;
u16 max_resolution;
BitmapForcedFormat force_bitmap_format;
SignedDistanceFieldMethod sdfgeneration;
s8 target_platform;
u16 spread_factor;
u16 generated_pad8ed2;
char usage_override[16];
s32 usage_override_count;
BitmapUsage bitmap_usage[usage_override_count];
char manual_sequence[16];
s32 manual_sequences_count;
BitmapGroupSequence manual_sequences[manual_sequences_count];
char source_data[24];
char source_path[20];
u64 source_checksum;
RealARgbColor min_color;
RealARgbColor max_color;
char sequence[16];
s32 sequences_count;
BitmapGroupSequence sequences[sequences_count];
char bitmapstart[16];
s32 bitmap_block_count;
BitmapBlock bitmap_block[bitmap_block_count];
char RawDDSData[header.ActualResoureDataSize];
};
struct InternalStruct {
s64 vtablespace;
u32 global_tag_id;
char local_tag_handle[4];
};
// TAG LAYOUT
Header header @ 0x00;
Tag_Dependency Dependencies[header.DependencyCount] @ 0x50;
Data_Block datablocks[header.DataBlockCount] @ header.DependencyCount * 0x18 + 0x50;
Tag_Def_Structure tagdefstructure[header.TagStructCount] @ header.DependencyCount * 0x18 + 0x50 + header.DataBlockCount * 0x10;
Data_Reference Data_References[header.DataReferenceCount] @ header.DependencyCount * 0x18 + 0x50 + header.DataBlockCount * 0x10 + header.TagStructCount * 0x20;
Tag_Fixup_Reference tagfixupreference[header.TagReferenceCount] @ header.DependencyCount * 0x18 + 0x50 + header.DataBlockCount * 0x10 + header.TagStructCount * 0x20 + header.DataReferenceCount * 0x14;
char ZoneSet[header.ZoneSetDataSize] @ header.HeaderSize - header.ZoneSetDataSize; // zonesets kinda make no sense
InternalStruct internalstruct @ header.HeaderSize;
Bitm bitmap @ header.HeaderSize + 16;

View File

@@ -0,0 +1,77 @@
#pragma author Surasia
#pragma description Halo Infinite Module
#pragma array_limit 4294967295
#pragma pattern_limit 4294967295
import std.string;
struct ModuleHeader
{
char magic[4];
s32 version;
s64 moduleId;
u32 fileCount;
s32 manifest0Count;
s32 manifest1Count;
s32 manifest2Count;
s32 resourceIndex;
u32 stringSize;
u32 resourceCount;
u32 blockCount;
u64 buildVersion;
s64 hd1Delta;
u64 dataSize;
};
struct ModuleFile
{
u32 resourceCount;
s32 parentIndex;
s16 pad;
u16 blockCount;
s32 blockIndex;
s32 resourceIndex;
char class[4] [[format("std::string::reverse")]];
u64 dataOffsetBytes;
u64 dataOffset = dataOffsetBytes & 0x0000FFFFFFFFFFFF [[export]];
u16 dataOffsetFlags = dataOffsetBytes >> 48 [[export]];
u32 totalCompressedSize;
u32 totalUncompressedSize;
u32 globalTagId;
u32 uncompressedHeaderSize;
u32 uncompressedTagDataSize;
u32 uncompressedResourceDataSize;
u32 uncompressedActualResourceDataSize;
u32 resourceBlockCount;
u32 nameOffset;
s32 parentResource;
u64 assetChecksum;
u64 assetId;
};
struct ModuleBlock
{
u32 compressedOffset;
u32 compressedSize;
u32 decompressedOffset;
u32 decompressedSize;
bool compressed;
s24 pad;
};
struct Module
{
ModuleHeader module_header;
ModuleFile files[module_header.fileCount];
$ = $ + 8;
s32 resources[module_header.resourceCount];
ModuleBlock blocks[module_header.blockCount];
padding[while($[$] == 0)];
std::print("Compressed Tag Data starts at: {}", $);
};
Module module_root @ 0x00;

168
patterns/hinf_tag.hexpat Normal file
View File

@@ -0,0 +1,168 @@
#pragma author Surasia
#pragma description Halo Infinite Tag File
import std.string;
import type.guid;
u32 fileStart = 0;
enum TagSectionType : u16
{
header,
tagData,
resourceData
};
enum TagStructType : u16
{
mainStruct,
tagBlock,
resource,
custom,
literal
};
struct TagHeader
{
char magic[4];
s32 version;
u64 hash;
u64 checksum;
u32 dependencyCount;
u32 dataBlockCount;
u32 tagStructCount;
u32 dataReferenceCount;
u32 tagReferenceCount;
u32 stringTableSize;
u32 zoneSetDataSize;
u32 unknown0;
u32 headerSize;
fileStart = headerSize;
u32 dataSize;
u32 resourceDataSize;
u32 actualResourceDataSize;
u8 headerAlignment;
u8 tagDataAlignment;
u8 resourceDataAlignment;
u8 actualResourceAlignment;
u32 unknown1;
};
struct TagDependency
{
char tagGroup[4] [[format("std::string::reverse")]];
u32 nameOffset;
u64 assetId;
u32 globalId;
u32 parentTag;
};
struct TagDataBlock
{
u32 entrySize;
u16 pad;
TagSectionType section;
u64 offset;
if (section == TagSectionType::tagData)
{
char dataBlock[entrySize] @ fileStart + offset;
}
};
struct TagStruct
{
type::GUID guid;
TagStructType type;
u16 unknown0;
s32 targetIndex;
s32 fieldBlockIndex;
u32 fieldOffset;
};
struct TagDataReference
{
s32 parentStructIndex;
s32 unknown0;
s32 targetIndex;
s32 fieldBlockIndex;
u32 fieldOffset;
};
struct TagReference
{
s32 fieldBlockIndex;
u32 fieldOffset;
u32 nameOffset;
s32 dependencyIndex;
};
struct TagZonesetHeader
{
s32 version;
u32 zonesetCount;
u32 footerCount;
s32 parentCount;
};
struct TagZonesetInstanceHeader
{
s32 stringId;
u32 tagCount;
u32 parentCount;
u32 footerCount;
};
struct TagZonesetTag
{
u32 globalId;
s32 stringId;
};
struct TagZonesetInstance
{
TagZonesetInstanceHeader header;
TagZonesetTag tags[header.tagCount];
TagZonesetTag footers[header.footerCount];
s32 parents[header.parentCount];
};
struct TagZoneset
{
TagZonesetHeader zoneset_header;
TagZonesetInstance instances[zoneset_header.zonesetCount];
};
struct Tag
{
TagHeader header;
TagDependency dependencies[header.dependencyCount];
TagDataBlock data_blocks[header.dataBlockCount];
TagStruct tag_structs[header.tagStructCount];
TagDataReference data_references[header.dataReferenceCount];
TagReference tag_references[header.tagReferenceCount];
TagZoneset zonesets;
if (header.headerSize != $)
{
std::print("Header size is wrong, additional data in tag");
$ = header.headerSize;
std::print("Data offset now at: {}", $);
}
};
Tag tag_root @ 0x00;

View File

@@ -41,7 +41,8 @@ enum Marker : u8 {
APP13 = 0xED,
APP14 = 0xEE,
APP15 = 0xEF,
COM = 0xFE
COM = 0xFE,
UNKNOWN = 0X00
};
enum DensityUnit : u8 {
@@ -103,6 +104,15 @@ fn sof0_component_read(SOF0Component c) {
return std::format("({}, {}, {})", c.componentId, c.samplingFactors, c.quantizationTableId);
};
fn get_eoi_marker_position() {
u32 pos = std::mem::find_sequence_in_range(0, $, std::mem::size(), 0xFF);
while (std::mem::read_unsigned(pos + 1, 1) != 0xD9) {
pos = std::mem::find_sequence_in_range(0, pos + 1, std::mem::size() + 1, 0xFF);
}
return pos;
};
struct SOF0 {
u8 bitsPerSample;
u16 imageHeight, imageWidth;
@@ -121,30 +131,34 @@ struct SOS {
u8 startSpectralSelection;
u8 endSpectral;
u8 apprBitPos;
u8 image_data[std::mem::size() - $ - 2] [[sealed]];
u8 image_data[get_eoi_marker_position() - $] [[sealed]];
};
struct Segment {
type::Magic<"\xff"> magic;
Marker marker;
if (marker == Marker::SOI || marker == Marker::EOI) {
if (std::mem::read_unsigned($, 1) != 0xFF) {
Marker marker = Marker::UNKNOWN;
u8 data[get_eoi_marker_position() - $];
} else {
u16 length;
if (marker == Marker::APP0) {
APP0 data;
} else if (marker == Marker::APP14) {
APP14 data;
} else if (marker == Marker::COM) {
char data[length - sizeof(length)];
} else if (marker == Marker::SOF0) {
SOF0 data;
} else if (marker == Marker::SOS) {
SOS data;
type::Magic<"\xff"> magic;
Marker marker;
if (marker == Marker::SOI || marker == Marker::EOI) {
} else {
u8 data[length - sizeof(length)] [[sealed]];
u16 length;
if (marker == Marker::APP0) {
APP0 data;
} else if (marker == Marker::APP14) {
APP14 data;
} else if (marker == Marker::COM) {
char data[length - sizeof(length)];
} else if (marker == Marker::SOF0) {
SOF0 data;
} else if (marker == Marker::SOS) {
SOS data;
} else {
u8 data[length - sizeof(length)] [[sealed]];
}
}
}
} [[format_read("segment_read")]];
@@ -153,4 +167,4 @@ fn segment_read(Segment s) {
return std::format("{}", s.marker);
};
Segment segments[while(!std::mem::eof())] @ 0x00 [[hex::visualize("image", this)]];
Segment segments[while(!std::mem::eof())] @ 0x00 [[hex::visualize("image", this)]];

View File

@@ -18,15 +18,23 @@ fn format_string(auto string) {
return string.value;
};
fn format_fixed32(auto fp32) {
return fp32.integer + fp32.fraction / 65536.0;
};
fn format_fixed16(auto fp16) {
return fp16.integer + fp16.fraction / 256.0;
};
struct FixedPoint16 {
u8 integer;
u8 fraction;
};
} [[format("format_fixed16")]];
struct FixedPoint32 {
u16 integer;
u16 fraction;
};
} [[format("format_fixed32")]];
struct string {
char value[std::mem::find_sequence_in_range(0, $, std::mem::size(), 0x00) - $];
@@ -122,8 +130,8 @@ struct TrackHeaderBox : FullBox {
s16 volume;
u16 reserved_3;
s32 matrix[9];
u32 width;
u32 height;
FixedPoint32 width;
FixedPoint32 height;
};
struct DataEntryBox : FullBox {

View File

@@ -455,6 +455,33 @@ struct ImportsTable {
$ = addressof(this)+coffHeader.optionalHeader.directories[1].size;
};
struct DelayedImportsDirectory {
u32 attributes;
u32 name;
u32 moduleHandle;
u32 delayImportAddressTable;
u32 delayImportNameTable;
u32 boundDelayImportTable;
u32 unloadDelayImportTable;
u32 timeStamp;
};
struct DelayImportsStructure {
if (parent.delayImportsDirectoryTable[std::core::array_index()].delayImportNameTable > 0) {
ImportsLookup delayedLookupTable[while(std::mem::read_unsigned($, wordsize()) != 0)] @ parent.delayImportsDirectoryTable[std::core::array_index()].delayImportNameTable - relativeVirtualDifference();
}
if (parent.delayImportsDirectoryTable[std::core::array_index()].name > 0) {
char dllName[] @ parent.delayImportsDirectoryTable[std::core::array_index()].name - relativeVirtualDifference() [[format("formatNullTerminatedString")]];
}
} [[inline]];
// https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#delay-load-import-tables-image-only
struct DelayedImportsTable {
DelayedImportsDirectory delayImportsDirectoryTable[while(std::mem::read_unsigned($, 16) != 0)];
DelayImportsStructure delayImportsStructures[sizeof(delayImportsDirectoryTable)/sizeof(delayImportsDirectoryTable[0])] [[inline]];
$ = addressof(this)+coffHeader.optionalHeader.directories[1].size;
};
// General Resource Table things
fn formatNullTerminatedString16(str string) {
return "\"" + std::string::substr(string, 0, std::string::length(string)) + "\"";
@@ -1063,6 +1090,9 @@ struct Section {
if (dataDirectoryInSection[9]) {
TLSTable tlsTable @ coffHeader.optionalHeader.directories[9].rva - relativeVirtualDifference();
}
if (dataDirectoryInSection[13]) {
DelayedImportsTable delayedImportTable @ coffHeader.optionalHeader.directories[13].rva - relativeVirtualDifference();
}
}
clearBoolArray();

33
patterns/psafe3.hexpat Normal file
View File

@@ -0,0 +1,33 @@
// Password Safe V3
// Only a small part of the file is unencrypted.
#pragma endian little
#pragma description Password Safe V3
import std.mem;
import std.sys;
struct Prologue {
u8 SALT[32];
u32 ITER;
u8 HPP[32] [[name("H(P')")]];
u8 B1[16];
u8 B2[16];
u8 B3[16];
u8 B4[16];
u8 IV[16];
};
struct EOF {
u8 HMAC[32];
};
char magic[4] @ 0x00;
std::assert(magic == "PWS3", "Invalid file: bad tag!");
char marker[16] @ std::mem::size() - 48;
std::assert(marker == "PWS3-EOFPWS3-EOF", "Invalid file: bad end marker!");
Prologue prologue @ 0x04;
u8 HMAC[32] @ std::mem::size() - 32;
EOF eof @ std::mem::size() - 48;

View File

@@ -1,5 +1,5 @@
#pragma author Fl3ch4
#pragma description TODO
#pragma MIME image/webp
#pragma endian big

View File

@@ -1,5 +1,8 @@
#pragma author 5h4rrK
#pragma description ReFS-File-System
#pragma author 5h4rrK // https://www.github.com/5h4rrK
#pragma organization temabi0s // (https://bi0s.in)
#pragma description ReFS-File-System
#pragma array_limit 10000
import type.types.win32;
import type.guid;
@@ -25,19 +28,46 @@ enum BLOCK : u32 {
MSBPlus = 0x2b42534d
};
struct SELF_DESCRIPTOR{
u32 start = $;
u64 LCN1[[name("FirstLCN")]];
u64 LCN2[[name("SecondLCN")]];
u64 LCN3[[name("ThirdLCN")]];
u64 LCN4[[name("FourthLCN")]];
$ = $+2;
u8 checksumtype[[name("ChecksumType")]];
u8 checksumoffset[[name("ChecksumOffset")]];
u16 checksumlen[[name("CheckSumLength")]];
$ = $+2;
if(checksumtype == 2){
u64 chksum[[comment("Crc64"),name("CRC64-CheckSum") ]];}
else if(checksumtype == 1){
u32 chksum[[comment("Crc32"),name("CRC32-CheckSum") ]];
}
u32 rem = this.parent.lenselfdescriptor - ($ - start);
BYTE buff[rem];
};
struct META_HEADERS {
BLOCK Signature[[comment("Block Signature"), name("BlockSignature")]];
u32 unk[[comment("Fixed Value 0x02"), name("Unknown")]];
$ = $ + (0x8 + 0x10);
u32 zero[[name("AlwaysZero")]];
u32 volsig[[name("VolumeSignature")]];
u128 blockNo[[comment("Most Recent Block"), name("PageRecency")]];
u64 LCN1[[comment("MetaPage 1st LogicalClusterNumber"), name("FirstLCN")]];
u64 LCN2[[comment("MetaPage 2nd LogicalClusterNumber"), name("SecondLCN")]];
u64 LCN3[[comment("MetaPage 3rd LogicalClusterNumber"), name("ThirdLCN")]];
u64 LCN4[[comment("MetaPage 4th LogicalClusterNumber"), name("FourthLCN")]];
u64 _Objid;
u64 ObjId;
u64 tableidlo[[name("TableIdentifierHigh")]];
u64 tableidhi[[name("TableIdentifierLow")]];
};
struct OFFSET_ENTRIES{
u16 offset[[name("Offset")]];
u16 unk[[name("Ignore")]];
};
struct ATTRIBUTE {
u64 LCN1[[name("FirstLCN")]];
u64 LCN2[[name("SecondLCN")]];
@@ -49,52 +79,129 @@ struct ATTRIBUTE {
BYTE ZeroPadding[56][[name("Padding")]];
};
enum NodeType : BYTE {
InnerNode = 0x01,
RootNode = 0x02,
StreamNode = 0x03,
};
struct INDEX_HEADER {
u32 size[[name("DataStartOffset")]];
u32 dataendoff[[name("DataAreaEndOffset")]];
u32 freebuff[[name("FreeBuff")]];
BYTE height[[name("Height")]];
NodeType ntype[[name("NodeType")]];
u16 unused1[[name("UnUsed")]];
u32 keyindxoff[[name("KeyIndexOffset")]];
u32 keycount[[name("KeysCount")]];
u64 unused2[[name("UnUsed")]];
u32 end[[name("KeyIndexEnd")]];
u32 align[[name("Alignment")]];
$ = this.parent.start_pos + 0x50 + this.parent.rootsize + keyindxoff;
OFFSET_ENTRIES entries[keycount][[name("KeyEntries")]];
};
enum Flags : u16 {
RightMost = 0x02,
DeletedEntry = 0x04,
StreamIndexEntry = 0x40,
};
struct INDEX_ENTRY_STRUCTURE {
u32 start_pos = $;
u32 indxentlen[[name("IndxEntryLen")]];
u16 keyoff[[name("KeyOffset")]];
u16 keylen[[name("KeyLen")]];
Flags flag[[name("Flags")]];
u16 valoff[[name("ValOffset")]];
u16 vallen[[name("ValLen")]];
$ = start_pos + keyoff;
char key[keylen][[name("Key")]];
$ = start_pos + valoff;
char value[vallen][[name("Value")]];
$ = start_pos + indxentlen;
};
struct ROOT_NODE{
u32 start_pos = $;
META_HEADERS pagehdr[[name("PageHeader")]];
u32 rootsize[[name("RootSize")]];
u16 fixed[[name("Fixed(0x28)")]];
u16 unk1[[name("Unknown")]];
u16 unk2[[name("Unknown")]];
u16 unk3[[name("Unknown")]];
u16 schema[[name("TableSchema")]];
u16 unk4[[name("Unknown")]];
u16 schemadup[[name("TableSchema")]];
u16 unk5[[name("Unknown")]];
u16 unk6[[name("Unknown")]];
u16 unk7[[name("Unknown")]];
u64 noofextends[[name("ExtendCount")]];
u64 noofrows[[name("RowsCount")]];
char comps[rootsize - ($ - start_pos) + 0x50][[name("Components")]];
$ = (start_pos + 0x50 + rootsize);
INDEX_HEADER indxhdr[[name("IndexHeader")]];
// $ = start_pos + 0x50 + rootsize + indxhdr.keyindxoff;
$ = (start_pos + 0x50 + rootsize + indxhdr.size);
INDEX_ENTRY_STRUCTURE indxentrystruct[indxhdr.keycount][[name("IndexEntry")]];
};
u32 keeptrack = 0;
struct EntryArray {
u32 AttrOffsetEntry;
struct GLOBALROOTNODE {
u32 GlobalElemEntryOff;
u32 prev = $;
$ = (0x1000 * ($ / 0x1000));
$ = $ + AttrOffsetEntry;
$ = $ + GlobalElemEntryOff;
keeptrack += 1;
if (keeptrack == 1) {
ATTRIBUTE ObjectTable;
ATTRIBUTE ObjectIDTable;
}
else if (keeptrack == 2) {
ATTRIBUTE UNKNOWN1;
ATTRIBUTE MediumAllocatorTable;
}
else if (keeptrack == 3) {
ATTRIBUTE UNKNOWN2;
ATTRIBUTE ContainerAllocatorTable;
}
else if (keeptrack == 4) {
ATTRIBUTE AttributeList;
ATTRIBUTE SchemaTable;
}
else if (keeptrack == 5) {
ATTRIBUTE DirectoryTree;
ATTRIBUTE ParentChildTable[[comment("Parent Child Table")]];
}
else if (keeptrack == 6) {
ATTRIBUTE UNKNOWN3;
ATTRIBUTE ObjectIDDup;
}
else if (keeptrack == 7) {
ATTRIBUTE UNKNOWN4;
ATTRIBUTE BlockCountTable;
}
else if (keeptrack == 8) {
ATTRIBUTE ContainerTable;
u32 prev_pos = $;
ROOT_NODE node1 @ (ContainerTable.LCN1 * clustersz)[[name("ContainerNode")]];
$ = prev_pos;
}
else if (keeptrack == 9) {
ATTRIBUTE ContainerTableDup;
u32 prev_pos = $;
ROOT_NODE node1 @ (ContainerTableDup.LCN1 * clustersz)[[name("ContainerDupNode")]];
$ = prev_pos;
}
else if (keeptrack == 10) {
ATTRIBUTE UNKNOWN5;
ATTRIBUTE SchemaTableDup;
}
else if (keeptrack == 11) {
ATTRIBUTE AllocatorLarge;
ATTRIBUTE ContainerIndexTable;
}
else if (keeptrack == 12) {
ATTRIBUTE UNKNOWN6;
ATTRIBUTE IntegrityStateTable;
}
else if (keeptrack == 13) {
ATTRIBUTE UNKNOWN7;
ATTRIBUTE SmallAllocatorTable;
u32 prev_pos = $;
ROOT_NODE node1 @ (SmallAllocatorTable.LCN1 * clustersz)[[name("SmallAllocatorNode")]];
$ = prev_pos;
}
$ = prev;
};
@@ -103,29 +210,31 @@ struct CHECKPOINT {
META_HEADERS CheckPointMetaHeader[[name("FSPageMetaHeader")]];
$ += (0x04);
CheckPoint_REFS_Version ReFSVersion;
u32 EntryOffset;
u32 EntrySz[[name("EntrySize")]];
u64 blockno[[name("BlockNumber")]];
u32 offsetselfdescriptor[[comment("Self Descriptor Offset")]];
u32 lenselfdescriptor[[comment("Self Descriptor Size"),name("SelfDescriptorSz")]];
u64 blockno[[comment("Most Recent CheckPoint") , name("BlockNumber")]];
u32 prev_pos = $;
$ = ($ / clustersz) *clustersz + offsetselfdescriptor;
SELF_DESCRIPTOR selfdes[[name("SelfDescriptor")]];
$ = prev_pos;
$ += (0x28);
u32 NumOfEntries;
EntryArray AttributeEntries[NumOfEntries];
GLOBALROOTNODE GlobalRootNodes[NumOfEntries];
$ += (0x08);
u64 LCN1dup[[comment("Duplicate LCN of MetaPage of this block"), name("DupFirstLCN")]];
u64 LCN2dup[[comment("Duplicate LCN of MetaPage of this block"), name("DupSecondLCN")]];
u64 LCN3dup[[comment("Duplicate LCN of MetaPage of this block"), name("DupThirdLCN")]];
u64 LCN4dup[[comment("Duplicate LCN of MetaPage of this block"), name("DupFourthLCN")]];;
};
struct SUPERBLOCK {
META_HEADERS SuperBlockMetaHeader[[name("FSPageMetaHeader")]];
type::GUID GUID;
$ = $ + (0x10 * 0x06);
$ = $ + (0x10 * 0x01);
u32 checkpointOffset[[name("CheckPointOffset")]];
u32 noofcheckpoint[[name("NoOfCheckPointEntry")]];
u32 offsetselfdescriptor[[name("SelfDescriptorOffset")]];
u32 lenselfdescriptor[[name("SelfDescriptorLength")]];
$ = $ + (0x10 * 0x04);
u64 primarychekpoint[[name("PrimaryCheckPoint")]];
u64 secondaychekpoint[[name("SecondaryCheckPoint")]];
u64 LCN1dup[[name("DupFirstLCN")]];
u64 LCN2dup[[name("DupSecondtLCN")]];
u64 LCN3dup[[name("DupThirdLCN")]];
u64 LCN4dup[[name("DupFourthLCN")]];
SELF_DESCRIPTOR selfdes[[name("SelfDescriptor")]];
};
struct VOLUME_BOOT_RECORD {
@@ -141,22 +250,37 @@ struct VOLUME_BOOT_RECORD {
ReFS_Version ReFSVersion;
BYTE UnknownBuff[0x0e][[name("Unknown")]];
u64 SerialNo[[name("SerialNumber")]];
u64 BytesPerContainer[[name("BytesPerContainer")]];
};
struct REFS_FILE_SYSTEM {
u64 checkVal = std::mem::read_unsigned($+3, 8);
$ = 0;
if(checkVal == FILESYSTEM::ReFS){
VOLUME_BOOT_RECORD vbr @ 0x00[[name("VolumeBootRecord")]];
SUPERBLOCK SuperBlock @ (0x1e * 0x1000);
SUPERBLOCK SuperBlock1 @ (0x1e * 0x1000)[[name("SuperBlock1")]];
u32 cluster_size = vbr.BytesPerSec * vbr.SectorPerCluster;
CHECKPOINT PrimaryCheckPoint @(SuperBlock.primarychekpoint * 0x1000);
CHECKPOINT PrimaryCheckPoint @(SuperBlock1.primarychekpoint * cluster_size)[[name("PrimaryCheckPoint1")]];
keeptrack = 0;
CHECKPOINT SecondaryCheckPoint @(SuperBlock.secondaychekpoint * 0x1000);
CHECKPOINT SecondaryCheckPoint @(SuperBlock1.secondaychekpoint * cluster_size)[[name("SecondaryCheckPoint1")]];
SUPERBLOCK SuperBlock2 @(((vbr.TotalNoOfSectors / vbr.SectorPerCluster) - 3) * cluster_size)[[name("SuperBlock2")]];
keeptrack = 0;
CHECKPOINT PrimaryCheckPoint2 @(SuperBlock2.primarychekpoint * cluster_size)[[name("PrimaryCheckPoint2")]];
keeptrack = 0;
CHECKPOINT SecondaryCheckPoint2 @(SuperBlock2.secondaychekpoint * cluster_size)[[name("SecondaryCheckPoint2")]];
SUPERBLOCK SuperBlock3 @(((vbr.TotalNoOfSectors / vbr.SectorPerCluster) - 2) * cluster_size)[[name("SuperBlock3")]];
keeptrack = 0;
CHECKPOINT PrimaryCheckPoint3 @(SuperBlock3.primarychekpoint * cluster_size)[[name("PrimaryCheckPoint3")]];
keeptrack = 0;
CHECKPOINT SecondaryCheckPoint3 @(SuperBlock3.secondaychekpoint * cluster_size)[[name("SecondaryCheckPoint3")]];
}
else{
break;
exit(0);
}
};
u32 clustersz = std::mem::read_unsigned($ + 0x20, $+4) * std::mem::read_unsigned($ + 0x24, $+4);
REFS_FILE_SYSTEM ReFSFileSystem @0x00;

726
patterns/ttf.hexpat Normal file
View File

@@ -0,0 +1,726 @@
#pragma author Shimogawa (aka Rebuild)
#pragma description TrueType and OpenType font format
#pragma endian big
#pragma MIME font/ttf
#pragma MIME font/otf
#pragma pattern_limit 1000000
import std.mem;
import std.string;
import std.time;
import std.io;
struct TableDirectory {
char tag[4];
u32 checkSum;
u32 offset;
u32 length;
};
fn format_fixed32(auto fp32) {
return fp32.integer + fp32.fraction / 65536.0;
};
fn format_f2dot14(auto f) {
return f.integer + f.fraction / 16384.0;
};
using FWord = s16;
using UFWord = u16;
struct FixedPoint32 {
u16 integer;
u16 fraction;
} [[format("format_fixed32")]];
bitfield F2Dot14 {
integer : 2;
fraction : 14;
} [[format("format_f2dot14")]];
fn format_longdatetime(auto t) {
return std::time::format(std::time::to_utc(u32(t.time - 2082844800)));
};
struct LongDatetime {
s64 time;
} [[format("format_longdatetime")]];
using HeadTable;
u64 currentOffset;
u64 currentLength;
u64 glyfStart;
bool hasPostScriptOutlines = false;
u16 numHMetrics;
u16 gNumGlyphs;
HeadTable headTable;
// begin avar
struct AxisValueMap {
F2Dot14 fromCoord;
F2Dot14 toCoord;
};
struct ShortFracSegment {
u16 positionMapCount;
AxisValueMap axisValueMaps[positionMapCount];
};
struct AvarTable {
FixedPoint32 version;
s32 axisCount;
ShortFracSegment segments[axisCount];
};
// end avar
// begin cmap
u64 startOfCmapTable = 0;
struct BaseCmapSubtable {
u64 start = $;
u16 format;
};
struct CmapSubtable0 : BaseCmapSubtable {
u16 length;
u16 language;
u8 glyphIndexArr[256];
};
struct CmapSubtable2 : BaseCmapSubtable {
u16 length;
u16 language;
u16 subHeaderKeys[256];
u8 data[length - 516];
};
struct CmapSubtable4 : BaseCmapSubtable {
u16 length;
u64 end = start + length;
u16 language;
u16 segCountX2;
u16 segCount = segCountX2 / 2;
u16 searchRange;
u16 entrySelector;
u16 rangeShift;
u16 endCode[segCount];
u16 reserved;
u16 startCode[segCount];
s16 idDelta[segCount];
u16 idRangeOffset[segCount];
u16 glyphIdArr[(end - $) / sizeof(u16)];
};
struct CmapSubtable6 : BaseCmapSubtable {
u16 length;
u16 language;
u16 firstCode;
u16 entryCount;
u16 glyphIdArr[entryCount];
};
struct SequentialMapGroup {
u32 startCharCode;
u32 endCharCode;
u32 startGlyphId;
};
struct CmapSubtable8 : BaseCmapSubtable {
u16 reserved;
u32 length;
u32 language;
u8 is32[8192];
u32 numGroups;
SequentialMapGroup groups[numGroups];
};
struct CmapSubtable10 : BaseCmapSubtable {
u16 reserved;
u32 length;
u64 end = start + length;
u32 language;
u32 startCharCode;
u32 numChars;
u16 glyphIdArr[(end - $) / sizeof(u16)];
};
struct CmapSubtable12 : BaseCmapSubtable {
u16 reserved;
u32 length;
u32 language;
u32 numGroups;
SequentialMapGroup groups[numGroups];
};
struct CmapSubtable {
u16 format = std::mem::read_unsigned($, 2, std::mem::Endian::Big);
match (format) {
(0): CmapSubtable0 table [[inline]];
(2): CmapSubtable2 table [[inline]];
(4): CmapSubtable4 table [[inline]];
(6): CmapSubtable6 table [[inline]];
(8): CmapSubtable8 table [[inline]];
(10): CmapSubtable10 table [[inline]];
(12): CmapSubtable12 table [[inline]];
(_): BaseCmapSubtable table [[inline]];
}
} [[name(std::format("Subtable Format {}", format))]];
enum CmapPlatform : u16 {
Unicode = 0,
Macintosh = 1,
ISO = 2,
Microsoft = 3,
Custom = 4,
};
struct EncodingRecord {
CmapPlatform platformId;
u16 encodingId;
u32 offset;
u64 endEncodingRecord = $;
$ = startOfCmapTable + offset;
CmapSubtable subtable;
$ = endEncodingRecord;
};
struct CmapTable {
startOfCmapTable = $;
u16 version;
u16 numSubtables;
EncodingRecord encodingRecords[numSubtables];
};
// end cmap
// begin cvt
struct CvtTable {
u64 size = currentLength / sizeof(FWord);
FWord controlValues[size];
};
// end cvt
// begin gasp
bitfield RangeGaspBehavior {
padding : 12;
symmetricSmoothing : 1;
symmetricGridfit : 1;
doGray : 1;
gridFit : 1;
};
struct GaspRange {
u16 rangeMaxPPEM;
RangeGaspBehavior rangeGaspBehavior;
};
struct GaspTable {
u16 version;
u16 numRanges;
GaspRange gaspRanges[numRanges];
};
// end gasp
// begin glyf
struct GlyphTableHeader {
u64 startOffset = $;
s16 numContours;
FWord xMin;
FWord yMin;
FWord xMax;
FWord yMax;
};
bitfield OutlineFlag {
padding : 2;
ySameOrPositive : 1;
xSameOrPositive : 1;
repeat : 1;
yShort : 1;
xShort : 1;
onCurve : 1;
};
u32 pIdx = 0;
u32 curGlyfSize;
struct SimpleGlyphFlag {
pIdx += 1;
OutlineFlag flag;
if (flag.repeat) {
pIdx += 1;
u8 repeatTimes;
}
};
struct SimpleGlyphTable : GlyphTableHeader {
u16 endPtsOfContours[numContours];
u32 numPoints = numContours == 0 ? 0 : endPtsOfContours[numContours - 1];
u16 instructionLength;
u8 instructions[instructionLength];
pIdx = 0;
SimpleGlyphFlag flags[while(pIdx < numPoints)] [[single_color]];
u8 data[startOffset + curGlyfSize - $];
};
bitfield CompositeGlyphFlag {
padding : 3;
unscaledComponentOffset : 1;
scaledComponentOffset : 1;
overlapCompound : 1;
useMyMetrics : 1;
weHaveInstructions : 1;
weHaveA2x2 : 1;
weHaveAnXAndYScale : 1;
moreComponents : 1;
padding : 1;
weHaveAScale : 1;
roundXYToGrid : 1;
argsAreXYValues : 1;
arg1And2AreWords : 1;
};
bool componentGlyphHasMoreComponents;
bool componentGlyphHasInstruction;
struct ComponentGlyphRecord {
CompositeGlyphFlag flags;
u16 glyphIndex;
if (flags.arg1And2AreWords) {
FWord arg1;
FWord arg2;
} else {
u8 arg1;
u8 arg2;
}
if (flags.weHaveAScale) {
F2Dot14 scale;
} else if (flags.weHaveAnXAndYScale) {
F2Dot14 xScale;
F2Dot14 yScale;
} else if (flags.weHaveA2x2) {
F2Dot14 xScale;
F2Dot14 scale01;
F2Dot14 scale10;
F2Dot14 yScale;
}
componentGlyphHasMoreComponents = flags.moreComponents;
componentGlyphHasInstruction = flags.weHaveInstructions;
};
struct CompositeGlyphTable : GlyphTableHeader {
componentGlyphHasMoreComponents = true;
componentGlyphHasInstruction = false;
ComponentGlyphRecord records[while (componentGlyphHasMoreComponents)];
if (componentGlyphHasInstruction) {
u16 instructionLength;
u8 instructions[instructionLength];
}
};
struct GlyfTable {
s16 type = std::mem::read_signed($, 2, std::mem::Endian::Big);
if (type >= 0) {
SimpleGlyphTable table [[inline]];
} else {
CompositeGlyphTable table [[inline]];
}
} [[name(std::format("glyf ({})", table.numContours < 0 ? "Composite" : "Simple"))]];
// end glyf
// begin hdmx
struct DeviceRecord {
u8 pixelSize;
u8 maxWidth;
u8 widths[gNumGlyphs];
};
struct HdmxTable {
u16 version;
u16 numRecords;
u32 sizeDeviceRecord;
DeviceRecord records[numRecords];
};
// end hdmx
// begin head
bitfield HeadFlag {
padding : 1;
lastResort : 1;
clearTypeOptimized : 1;
converted : 1;
lossless : 1;
indicStyleRearr : 1;
RTLGlyph : 1;
AATFont : 1;
linguisticRender : 1;
padding : 1;
verticalLayout : 1;
instrMayAlterAW : 1;
integerScaling : 1;
differentPointSize : 1;
xLBlackBitIsLSB : 1;
yZeroIsBaseline : 1;
};
bitfield MacStyle {
padding : 9;
extended : 1;
condensed : 1;
shadow : 1;
outline : 1;
underline : 1;
italic : 1;
bold : 1;
};
struct HeadTable {
FixedPoint32 version;
FixedPoint32 fontRevision;
u32 checksumAdjustment;
u32 magic;
HeadFlag flags;
u16 unitsPerEm;
LongDatetime created;
LongDatetime modified;
FWord xMin;
FWord yMin;
FWord xMax;
FWord yMax;
MacStyle macStyle;
u16 lowestRecPPEM;
s16 fontDirectionHint;
s16 indexToLocFormat;
s16 glyphDataFormat;
};
// end head
// begin hhea
struct HheaTable {
FixedPoint32 version;
FWord ascent;
FWord descent;
FWord lineGap;
UFWord advanceWidthMax;
FWord minLeftSideBearing;
FWord minRightSideBearing;
FWord xMaxExtent;
s16 caretSlopeRise;
s16 caretSlopeRun;
s16 caretOffset;
s16 reserved[4];
s16 metricDataFormat;
u16 numberOfHMetrics;
numHMetrics = numberOfHMetrics;
};
// end hhea
// begin hmtx
struct LongHorMetric {
UFWord advanceWidth;
FWord lsb;
};
struct HmtxTable {
LongHorMetric hMetrics[numHMetrics];
FWord leftSideBearings[gNumGlyphs - numHMetrics];
};
// end hmtx
// begin loca
// loca contains offsets for glyf tables, so
// we put glyf arrays here also.
struct GlyfWithOffset<auto longOffset> {
if (longOffset) {
u32 offset;
u32 realOffset = offset;
u32 nextOff = std::mem::read_unsigned($, 4, std::mem::Endian::Big);
} else {
u16 offset;
u32 realOffset = u32(offset) * 2;
u32 nextOff = u32(std::mem::read_unsigned($, 2, std::mem::Endian::Big)) * 2;
}
curGlyfSize = nextOff - realOffset;
u64 prev = $;
$ = glyfStart + realOffset;
if (curGlyfSize != 0) {
GlyfTable glyf;
}
$ = prev;
};
struct LocaTable {
if (headTable.indexToLocFormat == 1) {
//u32 offsets[gNumGlyphs + 1];
GlyfWithOffset<true> glyfs[gNumGlyphs];
} else {
// u16 offsets[gNumGlyphs + 1];
GlyfWithOffset<false> glyfs[gNumGlyphs];
}
};
// end loca
// begin maxp
struct MaxpTable {
FixedPoint32 version;
u16 numGlyphs;
gNumGlyphs = numGlyphs;
if (!hasPostScriptOutlines) {
u16 maxPoints;
u16 maxContours;
u16 maxComponentPoints;
u16 maxComponentContours;
u16 maxZones;
u16 maxTwilightPoints;
u16 maxStorage;
u16 maxFunctionDefs;
u16 maxInstructionDefs;
u16 maxStackElements;
u16 maxSizeOfElements;
u16 maxComponentDepth;
}
};
// end maxp
// begin name
u64 curStorageOffset;
struct NameRecord {
u16 platformId;
u16 encodingId;
u16 languageId;
u16 nameId;
u16 length;
u16 stringOffset;
u64 cur = $;
$ = curStorageOffset + stringOffset;
char16 string[length / 2];
$ = cur;
};
struct LangTagRecord {
u16 length;
u16 langTagOffset;
u64 cur = $;
$ = curStorageOffset + langTagOffset;
char16 langTag[length / 2];
$ = cur;
};
struct NameTableV0 {
u64 startOfNameTable = $;
u16 version;
u16 count;
u16 storageOffset;
curStorageOffset = startOfNameTable + storageOffset;
NameRecord nameRecords[count];
};
struct NameTableV1 : NameTableV0 {
u16 langTagCount;
LangTagRecord langTagRecords[langTagCount];
};
struct NameTable {
u16 version = std::mem::read_unsigned($, 2, std::mem::Endian::Big);
match (version) {
(0): NameTableV0 table [[inline]];
(_): NameTableV1 table [[inline]];
}
};
// end name
// begin OS/2
struct OS2TableV0 {
u16 version;
FWord xAvgCharWidth;
u16 usWeightClass;
u16 usWidthClass;
u16 fsType;
FWord ySubscriptXSize;
FWord ySubscriptYSize;
FWord SubscriptXOffset;
FWord SubscriptYOffset;
FWord ySuperscriptXSize;
FWord ySuperscriptYSize;
FWord SuperscriptXOffset;
FWord SuperscriptYOffset;
FWord yStrikeoutSize;
FWord yStrikeoutPosition;
s16 sFamilyClass;
u8 panose[10];
u32 ulUnicodeRange[4];
u8 achVendID[4];
u16 fsSelection;
u16 usFirstCharIndex;
u16 usLastCharIndex;
FWord sTypoAscender;
FWord sTypoDescender;
FWord sTypoLineGap;
UFWord usWinAscent;
UFWord usWinDescent;
};
struct OS2TableV1 : OS2TableV0 {
u32 ulCodePageRange[2];
};
struct OS2TableV4 : OS2TableV1 {
FWord sxHeight;
FWord sCapHeight;
u16 usDefaultChar;
u16 usBreakChar;
u16 usMaxContent;
};
struct OS2TableV5 : OS2TableV4 {
u16 usLowerOpticalPointSize;
u16 usUpperOpticalPointSize;
};
struct OS2Table {
u16 version = std::mem::read_unsigned($, 2, std::mem::Endian::Big);
match (version) {
(0): OS2TableV0 table [[inline]];
(1): OS2TableV1 table [[inline]];
(2 | 3 | 4): OS2TableV4 table [[inline]];
(_): OS2TableV5 table [[inline]];
}
};
// end OS/2
// begin post
struct PostTableV1 {
u64 startOfTable = $;
FixedPoint32 version;
FixedPoint32 italicAngle;
FWord underlinePosition;
FWord underlineThickness;
u32 isFixedPitch;
u32 minMemType42;
u32 maxMemType42;
u32 minMemType1;
u32 maxMemType1;
};
struct PostTableV2 : PostTableV1 {
u16 numGlyphs;
u16 glyphNameIndex[numGlyphs];
std::string::SizedString<u8> stringData[while ($ < startOfTable + currentLength)];
};
struct PostTable {
u16 major = std::mem::read_unsigned($, 2, std::mem::Endian::Big);
u16 minor = std::mem::read_unsigned($ + 2, 2, std::mem::Endian::Big);
match (major, minor) {
(2, 0): PostTableV2 table [[inline]];
(_, _): PostTableV1 table [[inline]];
}
};
// end post
struct HiddenForPreprocessing {
u64 defStart = $;
TableDirectory td;
$ = td.offset;
currentOffset = td.offset;
currentLength = td.length;
match (td.tag) {
("CFF "): {
hasPostScriptOutlines = true;
}
("maxp"): MaxpTable table;
("head"): {
HeadTable table;
headTable = table;
}
("glyf"): {
glyfStart = $;
}
}
$ = defStart + sizeof(TableDirectory);
} [[hidden]];
struct Table {
u64 defStart = $;
TableDirectory td [[inline]];
$ = td.offset;
currentOffset = td.offset;
currentLength = td.length;
match (td.tag) {
("avar"): AvarTable table;
("bhed"): HeadTable table;
("cmap"): CmapTable table;
("cvt "): CvtTable table;
("gasp"): GaspTable table;
// ("glyf"): GlyfTable table;
// glyf will be contained in loca.
("glyf"): {}
("hdmx"): HdmxTable table;
("head"): HeadTable table;
("hhea"): HheaTable table;
("hmtx"): HmtxTable table;
("loca"): LocaTable table;
("maxp"): MaxpTable table;
("name"): NameTable table;
("OS/2"): OS2Table table;
("post"): PostTable table;
(_): u8 data[td.length];
}
$ = defStart + sizeof(TableDirectory);
} [[name(std::format("Table({})", td.tag))]];
struct TTF {
u32 scalarType;
u16 numTables;
u16 searchRange;
u16 entrySelector;
u16 rangeShift;
u64 start = $;
HiddenForPreprocessing hidden[numTables] [[hidden]];
$ = start;
Table tables[numTables];
};
TTF ttf @ 0x0;

View File

@@ -1,3 +1,5 @@
#pragma description TODO
import std.core;
import std.io;
import std.mem;
@@ -229,7 +231,7 @@ struct DeviceQualifierDescriptor {
bitfield OTGAttributes {
SRPSupport : 1;
HNPSupport : 1;
} [[right_to_left]];
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 2)]];
struct OTGDescriptor {
OTGAttributes bmAttributes;

View File

@@ -0,0 +1,75 @@
// Wintec TES file
//
// Wintec produced small GPS-Loggers that created TES files. Each file
// is a series of timestamps and coordinates. The format is exceedingly
// simple. The same 16 byte struct is repeated from start to finish.
//
// This would be a valid line:
// 01 00 31 F9 DE 60 A0 B1 9C 16 A0 AD 45 0E 88 00
// ----- ----------- ----------- ----------- -----
// flags timestamp latitude longitude altitude
//
// The line decodes to:
// flag 0x01 -> split mark (start of a new file)
// timestamp 2024-03-15 15:36:49
// latitude 37.936784°
// longitude 23.944746°
// altitude 136m
//
// Made with help from a (since deleted) GPSBabel definition
// https://github.com/GPSBabel/gpsbabel/blob/gpsbabel_1_8_0/wintec_tes.cc
#include "std/mem.pat"
#include "std/string.pat"
using WBTTimestamp;
// helper to format latitude and longitude in a human-readable form
// values are stored in an s32, but are actually scaled 1e-7 degrees.
fn decimalDegrees(s32 input) {
return double(input)/10000000;
};
// same as above, but outputs a string for readability
fn stringDegrees(s32 input) {
return std::format("{:f}°", double(input)/10000000);
};
// create the standard iso8601 format from any timestamp
fn iso8601(WBTTimestamp input) {
return std::format("20{:02d}-{:02d}-{:02d}T{:02d}:{:02d}:{:02d}Z",input.year, input.month, input.day, input.hours, input.minutes, input.seconds);
};
// This 16 bit field is barely used. Only these two are known.
bitfield WBTFlags {
split: 1 [[comment("start of a new file")]];
waypoint: 1 [[comment("waypoint button pressed")]];
padding: 14;
};
// The files store the timestamp in a somewhat uncommon 32 bit format
bitfield WBTTimestamp {
seconds: 6;
minutes: 6;
hours: 5;
day: 5;
month: 4;
year: 6;
} [[format("iso8601")]];
// This is what parses each line of the file. From the very beginning to the very end,
// all data is made up of this 16 byte struct.
struct WBTLine {
WBTFlags flags;
WBTTimestamp timestamp [[color("ff0000")]];
s32 lat [[color("007FFF"), format("stringDegrees")]];
s32 lon [[color("7F00FF"), format("stringDegrees")]];
s16 alt [[color("0000FF")]];
} [[hex::visualize("coordinates", decimalDegrees(lat), decimalDegrees(lon))]];
// parsing that whole file start to finish:
WBTLine lines[while(!std::mem::eof())] @ 0x00;

View File

@@ -1,5 +1,5 @@
#pragma author WerWolv
#pragma description Nintendo Switch XCI cardridge ROM
#pragma description Nintendo Switch XCI cartridge ROM
import std.core;

157
patterns/xgspak.hexpat Normal file
View File

@@ -0,0 +1,157 @@
import std.mem;
import std.core;
import type.time;
import type.magic;
//By LolHacksRule
/*
XGSPAK VX only works with games using XGS Engine VX (I think First Touch Games use multiple?), using different versions with them is likely a bad idea as devs can only use one of the three.
XGSPAKV1 can be used with ZLib or Raw content.
Works nearly perfectly on PAKs WITH and WITHOUT folders.
*/
/*
TODO:
Double check XGSPAKV0 from NFS Wii, it's probably similar?
Fix FTG V1 PAK if possible, uses V0 layout
*/
/*
Format info I used:
https://aluigi.altervista.org/bms/angry_birds_go.bms
https://aluigi.altervista.org/bms/angry_birds_starwars.bms
https://aluigi.altervista.org/bms/xpk2.bms
https://aluigi.altervista.org/bms/dls20.bms
*/
enum PakVersion : u8
{
Version0, //NFS Wii/Trilogy 3DS/DLS
Version1, //ABGO V1/ABTF, AB Console Ports
Version2 //ABGO V2+
};
enum CompressionType : u32
{
Raw, //NFS Wii/Trilogy 3DS/DLS
Zlib, //ABGO V1/ABTF, AB Console Ports
LZ4 //ABGO V2+
};
struct XGSPakHeader
{
if (std::mem::read_unsigned($, 1) > 2) //Assume BE
{
std::core::set_endian(std::mem::Endian::Big);
char magic[3];
PakVersion ver;
}
else
{
PakVersion ver;
char magic[3];
}
u32 folders;
u32 files; //-1 bcz init folder
u32 fileContentTableSize;
if (ver == PakVersion::Version2)
{
CompressionType compressionType;
}
};
struct XGSPakFolderBase
{
if (Head.ver == PakVersion::Version0)
{
char *name[] : u32[[pointer_base("offToStringTable")]];
u32 filesInFolder;
u32 subfolders;
u32 filesPos;
u32 foldersPos;
}
else
{
if (Head.magic == "XPK")
{
u32 dummy;
char *name[] : u32 [[pointer_base("offToStringTable")]];
u32 dummy2;
u32 filesPos;
u32 dummy3;
u32 foldersPos;
u32 filesInFolder;
u32 subfolders;
}
else
{
//u64 name;
char *name[] : u64 [[pointer_base("offToStringTable")]];
u64 filesPos;
u64 foldersPos;
u32 filesInFolder;
u32 subfolders;
}
}
};
struct XGSPakContentMeta
{
if (Head.ver == PakVersion::Version0)
{
char *name[] : u32 [[pointer_base("offToStringTable")]];
u32 decompFileSize;
u32 fileOff;
u32 compressed;
type::time32_t timestamp;
u32 compFileSize;
}
else
{
if (Head.magic == "XPK")
{
u32 dummy;
char *name[] : u32 [[pointer_base("offToStringTable")]];
u32 decompFileSize;
u32 fileOff;
u32 compressed;
type::time32_t timestamp;
u32 compFileSize;
u32 dummy2;
}
else
{
//u64 name;
char *name[] : u64 [[pointer_base("offToStringTable")]];
u32 decompFileSize;
u32 fileOff;
u32 compressed;
type::time32_t timestamp;
u64 compFileSize;
}
}
if (compressed)
{
u8 compressedFile[compFileSize] @ fileOff;
}
else
{
u8 file[decompFileSize] @ fileOff;
}
};
XGSPakHeader Head @ $;
//u32 offToStringTable_old = ((0x20*Head.folders)+(0x20*Head.files))+sizeof (Head); //Finicky but it works
fn offToStringTable(u128 offset) {
match (Head.ver)
{
(PakVersion::Version0): return ((0x14*Head.folders)+(0x18*Head.files))+sizeof (Head);
(PakVersion::Version1 | PakVersion::Version2): return ((0x20*Head.folders)+(0x20*Head.files))+sizeof (Head);
}
};
XGSPakFolderBase FolderBase[Head.folders] @ $;
XGSPakContentMeta ContentMeta[Head.files] @ $;

113
patterns/xgstexture.hexpat Normal file
View File

@@ -0,0 +1,113 @@
import type.magic;
//By LolHacksRule
enum XGSPlatform : u8
{
WIN32_OR_ANDROID, //Android sure but Win32 p much was only used in EOP (Edge of Perception)?
PS3, //PlayStation 3,
iOS_OR_PSP, //iPhone/iPad or PSP (confusion, I think PSP was renamed to iOS but idk)
REVOLUTION, //Nintendo Wii
XENON, //Xbox 360
iOS_IPHONE_LEGACY, //Legacy iPhone/iPad since ABGO V1
CTR, //Nintendo 3DS
PSP2, //PlayStation Vita
OSX, //Mac OS X
ANDROID_LEGACY, //Definitely Android since ABGO V1
METRO_OR_WINSTORE, //WP7?
CAFE, //Wii U
BLACKBERRY,
OGL, //?
ORBIS, //PlayStation 4
DURANGO, //Xbox One
WINPHONE, //PCOGL
WINPHONE_LEGACY, //WP8
APPLE_TV, //Apple TV
};
enum XGSTextureFormat : u16
{
RGB565_SWIZZLED, //AB SW Console, Vita/PS3 uses no swizzle
RGB565, //ABTF but supported in GO
RGBA4444, //On PS3 this is BGRA8888/ARGB8888?
RGBA4444_2_OR_RGBA8888_OR_LA88, //On PS4/X360/XENON this is ARGB8888, on Win this is RGBA8888, on current Android this is RGBA4444 OR RGBA8888 (ABTF2017 Chi?), on caFe/Wii U or legacy iOS this is RGBA8888, on current iOS it's both, on Orbis/PS4 it's BGRA8888, on Wii it's linear indexed 32BPP LA88 with BC swizzle, on legacy Android or ABGO v101 crashes
RGBA8888, //On GO v101, nothing is read
PALLETIZED_RGBA5A3, //Palletized RGB5A3, AB SW Wii, doesn't crash in EOP or ABGO v101 (Android) but doesn't return any data
//6, //crash on EOP and PS3 but not ABGO v101 but doesn't return any data
//7, //crash on EOP and PS3 but not ABGO v101 but doesn't return any data
AL88_OR_DXT1_OR_RGB565 = 8, //On Xbox 360/PS3 this is usually DXT, on Wii U, RGBA8888, on Android, this is usually AL88 unless the format is DXT, on 3DS, swizzled RGB565, on EOP, crash, on ABGO v101, nothing
UNKNOWN_DXT, //DXT3? no XGS games so far use this one, crashes on EOP, on ABGO v101, nothing
DXT5_SWIZZLED, //ABSW PS3/WinPhone/Legacy Android only? on ABGO v101, nothing
PVR_UNK, //2BPP no alpha? Crashes in ABGO v101
RGB888, //Greyscale? unused, crashes in EOP but works in ABGO v101
AL88,
//14, //8BPP something? Crash on EOP and PS3, on ABGO v101, nothing
PVRTC_2BPP_RGBA = 15, //Crash on PS3, on ABGO v101, nothing
PVRTC_4BPP_RGBA, //Crash on PS3 or is this 2bppRGBA, on ABGO v101, nothing
PVRTC_4BPP_RGBA_2, //May need to swap these two, crash on PS3, on ABGO v101, nothing
PVRTC_4BPP_RGBA_3, //Crashes on EOP and PS3, on ABGO v101, nothing
//19, //Crash on PS3 and EOP, on ABGO v101, nothing
//20, //Crash on PS3 and EOP, on ABGO v101, nothing
//21, //Crash on PS3 and EOP, on ABGO v101, nothing
RG88_OR_GR88 = 22, //Unused. On PS3, it's unswizzled GR88, crashes on EOP, on ABGO v101, nothing
L4, //Unused, crashes on EOP, on ABGO v101, nothing
DXT1_OR_ETC1, //On Android this is usually DXT1, old Android has this DXT1 swizzled, null on PS3, crash on EOP, on ABGO v101, nothing
LA44, //Crash on EOP, on ABGO v101, nothing
DXT5, //Crash on EOP, on ABGO v101, nothing
//?, //Crash on EOP, on ABGO v101, nothing, note starting here PS3 crashes
//?, //Crash on EOP, on ABGO v101, nothing
PVRTC_2BPP_RGBA_2 = 29, //on ABGO v101, nothing
PVRTC_4BPP_RGBA_V2, //on ABGO v101, nothing
PVRTC_4BPP_RGB_V3, //on ABGO v101, nothing
//? //on ABGO v101, nothing
//? //on ABGO v101, nothing
//? //on ABGO v101, nothing
ETC2_RGB = 35, //on ABGO v101, nothing
//? //on ABGO v101, nothing
ATC = 37, //BC Swizzled, on ABGO v101, nothing
//ATC_RGBA_Exp, //? on ABGO v101, nothing
ATCA_Interpolated = 39, //BC Swizzled, on ABGO v101, nothing
//? //on ABGO v101, nothing
RGBA5A3 = 250, //on ABGO v101, nothing
CMPR, //CMPR/DXT1 with different blocks, nothing on ABGO v101
ETC2_RGB_2_OR_ETC1, //On CTR, ETC1 Z-Order 1, works on ABGO v101 (ETC1)
ETC1_RGB_SPLITALPHA, //CTR only, nothing on ABGO v101
ATC_RGB, //Confirm, nothing on ABGO v101
ATC_RGBA_EXPLICIT, //Prob nothing on ABGO v101
ATC_RGBA_INTERPOLATED, //Prob nothing on ABGO v101
XGSTEXFMT_257, //Defined in ABGO v101 but idk what it is
//258-263 are unknown
DXT1_EXT = 264, //DXT1 EXT (?)
DXT3_EXT, //DXT3 EXT (?)
DXT5_EXT, //DXT1 EXT (?)
};
bitfield XGSTextureFlags
{
Mips : 2; Twiddle : 2; Normal : 2; Border : 2; CubeMap : 2; MipMapDebugColoring : 2; Volume3D : 2; AlphaChannelPresent : 2; VFlip : 2;
//Idk how this works
};
struct XGSTextureHeader
{
type::Magic"XGST"> magic;
u8 dataOffset;
XGSPlatform compilationPlatform; //Definitely platform, this has no harm when modifying (except in Edge of Perception) but does help with knowing platform specific compressions, usually
u16 headerSize; //?
//byte dmy;
u8 numMips;
//? start
XGSTextureFlags flags;
//? end
//char format[4];
XGSTextureFormat format;
u16 numColors; //Modifying this seems to have no known effect but idk
u16 width;
u16 height;
u16 width2; //No use modifying
u16 height2; //No use modifying
u32 paletteSize; //Modifying this affects the game badly, take care
u32 dataSize;
};
XGSTextureHeader Head @ $;
u8 Texture[Head.dataSize] @ $;

View File

@@ -47,11 +47,11 @@ bitfield Header {
fn validate_hdr_checksum(Header hdr) {
// Reassemble as a 2-byte big endian value
u16 value = (
(hdr.method << 8) +
(u8(hdr.method) << 8) +
(hdr.info << 12) +
(hdr.fcheck << 0) +
(hdr.fdict << 5) +
(hdr.flevel << 6)
(u8(hdr.flevel) << 6)
);
if (value % 31 == 0) {

View File

@@ -65,7 +65,7 @@ int main(int argc, char **argv) {
if (!runtime.executeString(patternFile.readString(), "<Source Code>")) {
fmt::print("Error during execution!\n");
if (const auto &hardError = runtime.getError(); hardError.has_value())
if (const auto &hardError = runtime.getEvalError(); hardError.has_value())
fmt::print("Hard error: {}:{} - {}\n\n", hardError->line, hardError->column, hardError->message);
return EXIT_FAILURE;

View File

@@ -39,8 +39,13 @@ int main(int argc, char **argv) {
// Setup Pattern Language Runtime
pl::PatternLanguage runtime;
bool hasDescription = false;
{
constexpr auto DummyPragmaHandler = [](const auto&, const auto&){ return true; };
auto DescPragmaHandler = [&hasDescription](const auto&, const auto&){
hasDescription = true;
return true;
};
runtime.setDataSource(0x00, testFile.getSize(),
[&](pl::u64 address, pl::u8 *data, size_t size) {
@@ -51,6 +56,7 @@ int main(int argc, char **argv) {
runtime.setDangerousFunctionCallHandler([]{ return true; });
runtime.setIncludePaths({ includePath });
runtime.addPragma("MIME", DummyPragmaHandler);
runtime.addPragma("description", DescPragmaHandler);
runtime.addDefine("__PL_UNIT_TESTS__");
runtime.setLogCallback([](auto level, const std::string &message) {
@@ -71,11 +77,17 @@ int main(int argc, char **argv) {
if (!runtime.executeString(patternFile.readString(), "<Source Code>")) {
fmt::print("Error during execution!\n");
if (const auto &hardError = runtime.getError(); hardError.has_value())
if (const auto &hardError = runtime.getEvalError(); hardError.has_value())
fmt::print("Hard error: {}:{} - {}\n\n", hardError->line, hardError->column, hardError->message);
return EXIT_FAILURE;
}
if (!hasDescription) {
fmt::print("No description pragma found in pattern file\n");
fmt::print("Please add a #pragma description <your description> tag to the pattern!\n");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.