Compare commits

...

238 Commits

Author SHA1 Message Date
WerWolv
4299243e95 includes/std: Added hex::dec::lz4_decompress() 2024-07-07 15:51:40 +02:00
Surasia
23b3c2b5d0 git: Fix hinf_tag and hinf_module links in readme (#268) 2024-07-07 13:44:04 +02:00
Surasia
da1c2687e8 patterns/hinf_*: Add Halo Infinite tag and module file patterns (#267)
* patterns/hinf: Add Halo Infinite Tag and Module patterns

* patterns/hinf_bitmap: remove pattern
2024-07-06 17:33:03 +02:00
WerWolv
18f968df5b includes/std: Drastically improved std::mem::MagicSearch, added find_string and find_string_in_range 2024-07-04 20:47:39 +02:00
WerWolv
f797c711ca includes/std: Optimize std::bit::popcount 2024-07-04 17:58:12 +02:00
WerWolv
5380877da4 tests: Added error message when no description is used 2024-07-04 08:28:20 +02:00
Marc Butler
bb46276bde patterns: Added Password Safe Version 3 pattern (#266)
* Add password safe V3

* Added description pragma

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2024-07-04 08:27:51 +02:00
Nik
4242869ac1 patterns/zlib: Cast bitfield enums to integers before doing math with them 2024-07-03 22:48:15 +02:00
WerWolv
8bdcd814a9 includes/std: Added std::unimplemented() 2024-07-02 18:00:52 +02:00
WerWolv
d7811ff5e9 patterns: Replaced old bitfield order attributes with new ones
Fixes #265
2024-07-02 18:00:41 +02:00
WerWolv
4d0b3e21bc includes/std: Added better documentation to std::mem::current_bit_offset() 2024-07-02 18:00:05 +02:00
WerWolv
acd6903b21 includes/std: Added crc8, crc16 and crc64 functions
Fixes #226
2024-07-02 17:59:40 +02:00
WerWolv
8d2a39f7d4 git: Added build folders to gitignore 2024-07-02 17:58:47 +02:00
Nik
65ef3a7783 includes/std: Removed cyclic dependency between std.mem and std.string 2024-07-01 23:47:09 +02:00
Nik
21a0b99eac includes/std: Added missing string include to std::mem 2024-06-29 23:25:58 +02:00
Rebuild
a33b68921d patterns: Added ttf/otf format (#263)
* Add ttf/otf format

* ttf: Put cff to preprocessing
2024-06-29 22:52:21 +02:00
Nik
a0bb71be58 includes/std: Added std::core::execute_function() 2024-06-29 11:53:43 +02:00
WerWolv
c7fc39ac19 git: Make sure test data is always treated as binaries 2024-06-26 20:59:41 +02:00
WerWolv
b540ead0ae includes/std: Added std::IIndexed as a nicer wrapper around std::core::array_index() 2024-06-25 22:52:57 +02:00
Nik
26878548f0 includes/std: Fixed NullString not including null terminator and not checking for string end correctly 2024-06-23 18:45:13 +02:00
C3pa
e4c9d86755 includes/std: Add some fixes from the WerWolv/Documentation repo (#262)
* Escape backslash in DOSPath docs

* Spelling correction for std\sys.pat

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2024-06-22 10:59:36 +02:00
WerWolv
2a3de1b705 includes: Tabs -> Spaces 2024-06-22 10:46:30 +02:00
WerWolv
e02280f9ee includes/type: Added arbitrarily formattable type 2024-06-20 21:21:42 +02:00
WerWolv
bbba68cef7 includes/std: Added null-terminated strings to the strings library 2024-06-20 21:21:30 +02:00
Joachim Schiele
28b281b403 patterns/pe: Added .didata section support (#257) 2024-06-16 15:23:01 +02:00
Kirill Gladkikh
c807959d75 patterns/jpeg: Support EOF in the middle of JPEG (#240)
* Support EOF in the middle of file

* Changed to eoi marker searching
2024-06-16 15:16:01 +02:00
Francisco J. Solis
3416d30f2b pattern/dds: Fix bitfield PixelFormatFlags (#259)
* Fix bitfield PixelFormatFlags

The padding names should not repeat
Comments should be added to each field for clarification
Padding should be corrected for luminance.

* Re-work of PixelFormatFlags

Add more information considering current GIMP repository structure
https://gitlab.gnome.org/GNOME/gimp/-/blame/master/plug-ins/file-dds/dds.h\#L145
2024-06-09 22:15:05 +02:00
iTrooz
10bf1c76cf chore: mention description pragma in PR template 2024-05-20 01:05:48 +02:00
iTrooz
5b7c212029 git: use ccache (#253) 2024-05-20 01:01:59 +02:00
iTrooz
1c2e948940 test: Make description pragma mandatory (#251) 2024-05-20 00:35:24 +02:00
iTrooz
7ea34e410a fix: add missing ; to core.pat 2024-05-20 00:13:52 +02:00
iTrooz
56950e44d8 fix: use renamed PatternRuntime::getEvalError() (#252) 2024-05-20 00:10:34 +02:00
iTrooz
ed42452fc1 chore: remove old coverage code 2024-05-20 00:01:36 +02:00
iTrooz
a68ecb8888 fix: run tests concurrently 2024-05-20 00:01:00 +02:00
WerWolv
8d4cf59497 patterns/ffx: Updated submodule 2024-05-19 17:05:56 +02:00
Benjamin Gilbert
3c2fed22e4 patterns: Add Windows animated cursor (#242)
* patterns: Add Windows animated cursor

* Added readme entry

* Added mime type to readme

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2024-05-19 16:58:03 +02:00
Christoph Koehler
ff550bd105 patterns: Add Digital Terrain Elevation Data (DTED) pattern file (#243)
Tested on DTED0 and DTED2, will likely work with DTED1 as well.
2024-05-19 16:55:30 +02:00
Rebuild
0ad6e3abde patterns/mp4: Use FixedPoint32 instead of u32 for width and height (#248)
Update mp4.hexpat

* Use FixedPoint32 instead of u32 for width and height
* Add formatting for displaying fixed point 16.16 and 8.8 numbers
2024-05-19 16:55:04 +02:00
Marijn Kneppers
891968db53 includes/std: Add create_directories to std (#250) 2024-05-19 16:46:05 +02:00
Marijn Kneppers
5799d1d7ea includes/std: Make std::file::flush not delete files (#246) 2024-05-12 23:39:49 +02:00
Marijn Kneppers
9c6bfeb50f includes/hex: Added hex::core::add_virtual_file (#247) 2024-05-12 23:38:57 +02:00
AdventureT
9f7cdded6a Crashlvl: Added support for version 0.95 (#245)
* Working on new Version

* Needs testing

* Updated CrashLvl script to support version 0.94c

* Update README.md

* Added TestData and author to script

* Delete tests/patterns/test_data/CrashLvl.hexpat.bin

* CrashLvl.hexpat: Added support for version 0.95
2024-05-12 23:38:25 +02:00
Claudius Coenen
03298b0b0c patterns: Added Wintec TES file pattern (#238) 2024-04-21 12:33:19 +02:00
LolHacksRule
5f2738872e patterns: Added XGSPak and XGSTexture patterns (#237)
* Add XGSPak and XGSTexture patterns

* Whoops forgot proper pointers

* Fix typo

* Whoops

* Proper start

* Update xgstexture.hexpat
2024-04-21 12:32:26 +02:00
nazDridoy
63585e6de7 patterns/7z: Added MIME type (#241) 2024-04-21 12:25:00 +02:00
Sabhya Raj Mehta
cb6caafa64 patterns/refs: Added Structure for MinStoreB+ node, missing GlobalRootNode Names & parses all Superblocks (#236)
[+] Added Structure for MinStoreB+ node, Missing GlobalRootNode Names
2024-03-29 11:55:01 +01:00
Leon Jacobs
236fadee47 patterns: Added gguf pattern (#235)
* (feat) add gguf parser

* (chore) cleanup empty struct

* (tests) add sample gguf for tests and update readme

* (feat) update type enum
2024-03-28 21:55:18 +01:00
GlorifiedPig
0549e62a14 patterns/tar: Fixed minor typo (#230)
changes "if" to "id"
2024-03-21 22:16:37 +01:00
Sabhya Raj Mehta
b4bf2b946f patterns/refs: Added filesystem Checking, comments, Renamed REFS_HEADER, and Fixed typo. (#231)
Added FileSystem Checking and comments && renamed struct ReFSHeader to VolumeBootRecord && Fixed Typo
2024-03-21 22:16:15 +01:00
Zwip-Zwap Zapony
5e82bfadac includes/std: Replace get_attribute_value with get_attribute_argument (#233)
Update get_attribute_argument/get_attribute_value

Replace std::core::get_attribute_value with std::core::get_attribute_argument
Mark optional arguments with square brackets in the documentation comments, and list their default values
2024-03-21 21:48:53 +01:00
Sabhya
a5c9f3b18a patterns: Added ReFS filesystem pattern (#229)
Added Pattern File For Parsing ReFS filesystem Header, SuperBlock and CheckPoint && updated README.md
2024-03-17 10:19:18 +01:00
Nik
38262e6bb9 tests/patterns: Moved xilinx bootgen test file to correct folder 2024-03-16 10:24:09 +01:00
Nik
0a37b60d70 git: Mark all test files as binary 2024-03-16 10:22:51 +01:00
Shadlock0133
345e264ff8 patterns: Add .bgcode file (#222)
Support for binary G-code files, used by 3d printers
2024-03-15 21:54:58 +01:00
syuuha
ba2c396534 patterns: Added pattern for ADTS files (#223) 2024-03-15 21:54:41 +01:00
paxcut
439f04e19b includes/std: Fixed "unknown function fm" error was shown in pattern data. (#225) 2024-03-15 21:54:03 +01:00
ogmini
eaeea8d093 patterns: Added pattern for Windows 11 Notepad Window State bin file (#228)
* Add files via upload

* Update README.md

* Update README.md

* Add files via upload

* Update Notepad-WindowState.hexpat

* Update and rename Notepad-WindowState.hexpat to notepadwindowstate.hexpat

* Rename Notepad-WindowState.hexpat.bin to notepad-windowstate.hexpat.bin

* Update README.md
2024-03-15 21:53:40 +01:00
Nik
9af8d7e693 includes/std: Added back #pragma once to mem library 2024-02-27 20:28:02 +01:00
WerWolv
dac2e0f786 patterns: Fixed a few errors 2024-02-25 22:59:32 +01:00
Nik
f70b7066b9 includes: Switch over to import statements (#224)
* Update includes and patterns to new import system

* Update namespaces to new syntax
2024-02-25 22:04:41 +01:00
WerWolv
3f44a743e8 yara: Added advanced analysis rules 2024-02-25 11:29:59 +01:00
WerWolv
834e5261b3 tests: Fixed tests not building 2024-02-18 13:07:04 +01:00
Nik
0e2966f10d includes/std: Added missing std::core::set_pattern_comment function 2024-02-09 13:23:51 +01:00
Nik
b2bc248447 includes/hex: SectionId -> Section 2024-01-31 08:24:03 +01:00
Nik
e6589ecba1 includes/std: Make std::mem::read_string take in a larger size 2024-01-31 08:22:25 +01:00
apewo
f75ba4b6ee patterns/pe: Add extra MIME Type (#220)
add pe mime

Co-authored-by: John Doe <bla@bla.bla>
2024-01-29 18:36:21 +01:00
Nik
cf18580960 includes/hex: Added decompress functions 2024-01-27 17:05:08 +01:00
lucia c
e7d1973957 patterns: Added SWF header pattern (#219)
* patterns: Added pattern for SWF files.

* Added test file for SWF header.

* Update README with SWF file type.
2024-01-27 15:53:16 +01:00
t1hq
35cc68544b encodings: Add custom encoding file for Big5 (#218)
* add big5 encoding table

* add notes for big5 encoding, and how to import custom encoding file
2024-01-27 11:18:47 +01:00
Nik
41d8967deb patterns/pyc: Remove unused function 2024-01-20 22:14:47 +01:00
pew
658ec20b37 patterns: Add .ras pattern (#216)
* Add simple .ras file pattern

* Update ras.hexpat

* Update README.md

* Add .ras test file

* Update ras.hexpat

* Update ras.hexpat

* Update ras.hexpat

* Update ras.hexpat
2024-01-10 00:38:18 +01:00
paxcut
fb5fcbafc1 includes/type: Changes to magic type to include address of failed asserts. (#217) 2024-01-10 00:36:17 +01:00
Brandon Maier
6f7988e96e patterns: Added Xilinx boot images (#210)
* Adds pattern for Xilinx's Zynq UltraScale+ Boot Image format.
* Boot images are generated by Xilinx's `bootgen` tool.
* Spec: Xilinx UG1283 https://docs.xilinx.com/r/en-US/ug1283-bootgen-user-guide/Zynq-UltraScale-MPSoC-Boot-and-Configuration
* Add test file generated as follows
  - Build bootgen tool from https://github.com/Xilinx/bootgen
  - Create dummy data with `dd if=/dev/zero of=image.bin count=1`
  - Create bif with `echo "img: {image.bin}" >image.bif`
  - Create test bin with
    `./bootgen -arch zynqmp -image image.bif -o xilinx_bootgen.hexpat.bin`
2023-12-28 18:32:13 +01:00
g4ngli0s
0a631f0c1f patterns/pe: Fix BaseRelocationBlock count inside BaseRelocationTable (#212) 2023-12-28 18:31:53 +01:00
EvelynTSMG
5942897c29 patterns/ffx: Pull in latest updates (#209)
* Add Final Fantasy X patterns

* Update README

* Fix: github dum

* Add Bastion patterns

* git: update submodules

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2023-12-22 14:02:37 +01:00
utkumaden
a7e6209791 patterns: Added gITF pattern(#205)
* Create gltf.hexpat

Added glTF binary file pattern.

* Add a test file for the glTF pattern.

* Renamed the test file to fit the pattern file name.

* Fixed typo in documentation.
2023-12-07 17:09:25 +01:00
EvelynTSMG
294f69fa36 patterns: Add Bastion patterns (#207)
* Add Final Fantasy X patterns

* Update README

* Fix: github dum

* Add Bastion patterns

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2023-12-07 16:38:32 +01:00
gmestanley
1ea12cd4bd patterns/pe: Bringing over the massively improved version of pe.hexpat (#204)
* Add files via upload

* Update README.md

Added nes.hexpat to list of hex patterns

* Fix indenting of hex patterns list

Moved nes.hexpat below NE.hexpat

* Update README.md

Fixed indenting of PP pattern file's description being wrong

* Added x-msdownload MIME type to PE description

* Made indenting & code of ips.hexpat better

* Improvements to gb.hexpat

* Urgent fix to ips.hexpat

* Urgent fix to gb.hexpat

* Massive pe.hexpat improvement

* Replaced virtualDifference with relativeVirtualDifference

* Fixing unnecessary pattern_limit
2023-12-07 16:36:50 +01:00
Surasia
8129504dcb patterns/hinf_luas: Added pattern for reading HavokScript 5.1 Bytecode from Halo Infinite (#203)
* patterns/hinf_luas: Added pattern for reading HavokScript 5.1 Bytecode from Halo Infinite

* Parses Header and Structures of the parent tag
* Reads Bytecode types and arguments
* Reads Constants and Debug Info
* Reads Referenced Tags in the File

* readme: fix relative link
2023-12-07 16:36:27 +01:00
klirktag
22e30cfef9 patterns/mp4: Add STBL-box children (#202)
add STBL-box children
2023-12-07 16:36:07 +01:00
EvelynTSMG
9b819cf392 patterns: Add Final Fantasy X patterns (#197)
* Add Final Fantasy X patterns

* Update README

* Fix: github dum
2023-11-20 11:54:56 +01:00
klirktag
3277821153 patterns/mp4: Support mp4a and avc1 codec boxes (#195)
add support for mp4a and avc1 codec boxes, and sample-time-to-sample box
2023-11-15 22:26:10 +01:00
paxcut
f23dbbb565 includes/std: Fixed std::string::replace not replacing sequences correctly (#198)
fix:  [Bug] std::string::replace("aa", "aa", "") returns "a" #1408
2023-11-15 22:25:34 +01:00
Surasia
52f7d9e77a patterns/hinf_bitmap: Added pattern for parsing Halo Infinite bitmap files (#194)
* patterns/hinf_bitmap: Added pattern for parsing Halo Infinite bitmap files

* Parses the header of all Halo Infinite files which includes related "dependencies", datablocks, structure references
* Read bitmap sequence data, including bitmap index, complete index, sprite info
* Find the width/height and DDS Format of the raw texture, which can then be parsed into usable DDS textures
* Locates raw DDS block info

* Nulled out the DDS section of the bitmap
2023-11-14 00:20:32 +01:00
klirktag
99df77baf2 patterns/mp4: Add childbox support for STBL boxes (#192)
* find childboxes in stbl

* add implementation for SampleDescriptionBox
2023-11-14 00:06:50 +01:00
klirktag
d7b0819ff0 patterns/mp4: Make handler_type into a char[4], which reflects how it is used (#193)
make handler_type into a char[4], which reflects how it is used
2023-11-14 00:06:31 +01:00
Nik
70f491d2fa includes/type: Fixed formatting of time_t values in big endian patterns 2023-11-12 22:18:46 +01:00
Hugo
08680a6544 patterns/bplist: Added pattern file for Apple’s binary property list format (bplist) (#190)
* Added pattern file for Apple’s binary property list format (bplist)

* renamed some stuff and improved error messages

* added error handling for object size special case (0x0F)
2023-11-12 01:02:50 +01:00
Ange Albertini
23bc36c4bd patterns/selinux: Detailed SELinux config bitfield (#188) 2023-11-06 16:30:53 +01:00
Ange Albertini
8ae778944d patterns/se: Increased array limit (#187)
Increased array limit
2023-11-01 20:19:10 +01:00
klirktag
7e6a5d3bfa patterns/mp4: Added stbl box (#185) 2023-10-29 19:54:23 +01:00
Hugo
27e55d2e6d patterns/zip: Added parsing of extra field (#182)
* zip pattern: Improved fallback method for finding eocd. Added test data to cover this edge case

* zip pattern: added parsing of extra field
2023-10-29 19:47:11 +01:00
Benjamin Gilbert
e0a602c10a patterns/jpeg: Small improvements (#184)
* patterns/jpeg: exclude EOI marker from SOS image data

* patterns/jpeg: use type::Magic for magic numbers

* patterns/jpeg: add RGB component IDs

* patterns/jpeg: add struct for APP14 marker

* patterns/jpeg: add definition for COM marker

* patterns/jpeg: add some format_read() for better legibility
2023-10-29 19:46:51 +01:00
AdventureT
8748646016 patterns/CrashLvl: Update Crash Back in Time level format (#186)
* Working on new Version

* Needs testing

* Updated CrashLvl script to support version 0.94c

* Update README.md

* Added TestData and author to script

* Delete tests/patterns/test_data/CrashLvl.hexpat.bin
2023-10-29 19:46:28 +01:00
iTrooz
4a52ee4cf8 patterns: add UEFI Boot Entry pattern (#183)
add uefi_boot_entry pattern
2023-10-24 23:04:49 +02:00
iTrooz
e517e3534b patterns: fix description pragmas (#181) 2023-10-24 17:25:43 +02:00
DmitriLeon2000
edd0aa9a2f patterns: Add .fas and .was pattern files (Oska DeskMates) (#176)
* Add .fas and .was pattern files (Oska DeskMates)

* Update .was pattern file

* Update .was/.wa3 pattern file

* Update README.md

* Update README.md

* Update .fas & .was pattern files

* Update README.md

* Update fas_oskasoftware_old.hexpat

* Added WAS test file

* Update WAS test file

* Update was_oskasoftware.hexpat
2023-10-24 07:35:38 +02:00
Hugo
a992d1ba92 patterns/zip: Improved fallback method for finding eocd (#177)
zip pattern: Improved fallback method for finding eocd. Added test data to cover this edge case
2023-10-24 07:35:02 +02:00
leen
6fbdcac556 patterns/ip: Fix payload size when ip header or tcp header has options in ip.hexpat (#178) 2023-10-24 07:34:35 +02:00
itsmeow
47fce1628f patterns: Add ARIA2 Control format (#179) 2023-10-24 07:33:56 +02:00
Nik
9b152ae560 includes/std: Fixed doc comment of std::math::atan2 2023-10-22 21:25:14 +02:00
Nik
6b136b7fc6 includes/std: Fixed std::math::atan2 not taking correct arguments 2023-10-22 21:23:51 +02:00
gmestanley
93494a19a4 patterns: Better code + credit given for ips.hexpat & better readme (#174)
* Add files via upload

* Update README.md

Added nes.hexpat to list of hex patterns

* Fix indenting of hex patterns list

Moved nes.hexpat below NE.hexpat

* Update README.md

Fixed indenting of PP pattern file's description being wrong

* Added x-msdownload MIME type to PE description

* Made indenting & code of ips.hexpat better

* Improvements to gb.hexpat

* Urgent fix to ips.hexpat

* Urgent fix to gb.hexpat
2023-10-15 21:21:19 +02:00
H0L0
4fd710e23e patterns/flac: Fix SEEKTABLE metadata block (#173)
Add SEEKTABLE to blocktype check
2023-10-15 21:19:34 +02:00
Nik
44842dc44b includes/std: Added std::mem::set_section_size (#175) 2023-10-15 21:15:12 +02:00
HalfInner
056035c540 patterns/zip: Add parsing local file header with minor adjustment (#172)
patterns/zip: add parsing local file header, and another adjustmenst

Add missing parsing for local file headers
Add compression method
Minor adjustments: removed black colors, marked magic ids
2023-10-06 12:20:30 +02:00
Diego
5ad8e15afc patterns/shp: Added ESRI patterns (#170)
* add .shp and .shx patterns

* add tests
2023-10-06 12:20:11 +02:00
Nik
39c4d11404 patterns/uefi: Remove invalid MIME pragma 2023-10-05 08:54:59 +02:00
Benjamin Gilbert
815bd5f7f3 git: Minor readme fixes (#168) 2023-09-26 16:51:31 +02:00
Benjamin Gilbert
917b05a9f2 patterns: Add DICOM (#162)
DICOM files specify a "transfer syntax" which defines the image
compression format, the encoding rules for the entire file, and any
special-case encoding rules for particular fields.  Many transfer syntaxes
have common design features, but a transfer syntax is allowed to use
completely custom encoding rules, and need not be publicly documented.

Explicitly reject transfer syntaxes we know we don't support: implicit VR
(uncommon), explicit VR big-endian (deprecated), deflate-compressed.
In all other cases, follow some general rules and hope it works out.

The large lookup functions were generated by a script, but I haven't
included the code here, since its input is hand-massaged data copied from
HTML tables.

Add a JPEG WSI test case from the WG26 2021 hackathon (CC0 license) and a
synthetic LE Explicit VLP case produced with img2dcm.
2023-09-26 16:51:02 +02:00
brliron
46a2cef993 patterns: Added Android VBMeta image pattern (#169) 2023-09-26 14:03:24 +02:00
Nik
bca25b1a78 includes/hex: Fixed duplicate variable name in hex::core::get_selection() function
Fixes #167
2023-09-25 07:22:06 +02:00
Benjamin Gilbert
7ecd6d87dd patterns/tiff: Support BigTIFF and tiled TIFF; bugfixes and cleanups (#159)
* patterns/tiff: show TIFF tag name/number in DirectoryEntry array

Don't require unfolding the array entry to see what tag it contains.

* patterns/tiff: show IFD number in IFD list

* patterns/tiff: add JPEGTables tag; correct name of ICC Profile tag

* patterns/tiff: add TIFFRational type

Rationals, uniquely, are primitives with two fields.  Add a struct to
represent this, rather than inlining them.

* patterns/tiff: use correct types for fields smaller than 4 bytes

Small fields are always left-aligned in the 4-byte Value Offset.  On
little-endian TIFFs we currently cheat this by declaring a 32-bit value
and letting little-endianness handle the semantics.  However, this adds
some extra conditionals, and misrepresents the resulting field as 32 bits.
Drop the cheat.

* patterns/tiff: add ValueOffset abstraction

We were using the field type to make assumptions about whether the Value
Offset is a Value or an Offset, which is incorrect.  If the Count
multiplied by the field size is larger than 4, the field is an Offset;
otherwise it's a Value.

Add display sugar for single-element arrays to avoid extra nesting.

* patterns/tiff: drop dead code

get_ifds_offsets() and BIG/LITTLE aren't used at all.  get_total_IFDs()
is only used for declaring the length of TIFFFile.IFDs, and isn't needed
because IFDs are structured as a linked list.

* patterns/tiff: drop set_endian()

The call in TIFFFile is redundant.  Drop both calls and open-code the
check at the top level, before executing any code.  The BigTIFF check
will eventually be added alongside this one.

Fail if we don't recognize the magic number.

* patterns/tiff: minor cleanups

* patterns/tiff: drop strip offset/byte count arrays from struct IFD

They're redundant with the fields in the DirectoryEntry array.  Also
they're buggy: they assume the field Value Offsets are always offsets,
which isn't true for single-strip IFDs, and they ignore a partial last
strip in multiple-strip IFDs.

* patterns/tiff: rewrite strip array generation

We're making extra work for ourselves by avoiding the type system.  Also,
by calculating the number of strips we expect rather than the number of
strips we actually have, we're miscounting and omitting any partial last
strip.

Instead, read the strip offsets and byte counts directly from the
IFDEntry array.

* patterns/tiff: add ImageData array for tiled IFDs

* patterns/tiff: increase variable widths for BigTIFF

Use 64-bit temporary variables for values that can be 64 bits in BigTIFF.

* patterns/tiff: support BigTIFF

* tests/patterns/test_data: add more TIFF tests
2023-09-24 20:04:20 +02:00
dn-ln
bf0d96db5f patterns/id3: Fix the error No variable named 'n' found. (#160) 2023-09-24 20:03:49 +02:00
paxcut
705900d38b includes/std: Fixed std::string::replace (#163)
The submitted fix may have been a work in progress version instead of the tested solution I intended to submit. Hopefully this time I'll get it right.
2023-09-24 19:58:04 +02:00
gmestanley
734afdf500 patterns: Added .nes pattern file (#166)
* Add files via upload

* Update README.md

Added nes.hexpat to list of hex patterns

* Fix indenting of hex patterns list

Moved nes.hexpat below NE.hexpat
2023-09-24 19:57:20 +02:00
Benjamin Gilbert
c0c117ac19 tests/patterns: support multiple test files for a pattern (#161)
Allow either a single test file in tests/patterns/${pattern}.hexpat.${ext},
as we do now, or multiple files in tests/patterns/${pattern}.hexpat/*.

Closes #158.
2023-09-10 09:56:25 +02:00
paxcut
7fba66a444 includes/std: Fixed std::string::replace truncating output incorrectly (#157) 2023-09-10 09:40:29 +02:00
Yury Ershov
8eed75d783 patterns/bson: Support for multiple documents per file (#156)
* BSON support multiple documents per file

BSON files can contain consecutive documents glued one after another. An example of these is MongoDB FTDC metrics export.

[`bsondump`](https://github.com/mongodb/mongo-tools/blob/master/bsondump/bsondump.go) can unpack this type of BSON documents.

* Add accidentally deleted lines
2023-09-10 09:39:32 +02:00
Kenichi Saita
81f4978656 patterns/vhdx: Various improvements (#154)
* Replaced right_to_left to bitfield_order

* Fixed type of ParentLocatorEntry (char -> char16)

* Fixed item address of MetadataTableEntry

* Improved RegionTableEntry# Please enter the commit message for your changes. Lines starting
2023-09-10 09:39:04 +02:00
iTrooz
b6e0557a1d patterns: put author and description inside each pattern (#155) 2023-08-28 12:32:23 +02:00
gmestanley
ba14dd0cb2 patterns: Adding IPS pattern and fixed PE magic file (#153)
* Update pe.hexpat

New improvement

* Add ips.hexpat via upload

* Add ips.hexpat.ips via upload

* Added IPS to README

* Mentioned Windows in portable_executable_magic
2023-08-27 00:38:30 +02:00
MatrixEditor
86f93dfdaf patterns: Added Ubiquiti firmware pattern (#145)
* Ubiquiti firmware pattern

* Added padding and crc32 fields

* Added unit test file

+ Renamed fields in partition struct
2023-08-16 23:28:01 +02:00
Justus Garbe
001900e3c2 patterns/pyc: Add a pattern for pyc files
Change name
2023-08-15 09:30:19 +02:00
Nik
6baae92553 includes/std: Added missing mem library include in ptr libary 2023-08-04 22:56:03 +02:00
Jason Shirk
520f9bcb22 patterns/pe: Fix DebugDirectory version field types (#146) 2023-08-04 01:14:49 +02:00
vikke1234
44b0392b78 patterns/fdt: Unflatten FDT (#144)
In order to have a better time viewing FDTs in the visualizer
scanner will now unflatten the FDT.

It could still be improved slightly by modifying FDTProps to
make their values be `u32`s and possibly detect whether they
should be u64s.
2023-08-04 01:11:20 +02:00
Nik
df2bd4e3f9 git: Run PatternLanguage webpage CI 2023-07-29 12:12:33 +02:00
Nik
7723cf55c6 patterns: Fixed wrong extensions for some patterns 2023-07-24 16:14:37 +02:00
Nik
caa2b6aaa6 patterns/fdt: Improved visualizing of data 2023-07-24 16:13:34 +02:00
lopqto
7344df7ff6 patterns/pyinstaller: Added PyInstaller pattern (#142) 2023-07-16 20:21:59 +02:00
Nik
df649d2e62 patterns/elf: Speed up section name querying 2023-07-14 21:52:47 +02:00
Nik
8710b9e66f scripts: Added basic Kaitai to ImHex Pattern converter 2023-07-14 17:08:49 +02:00
vikke1234
665c942329 patterns/fdt: Fixed typo (#141) 2023-07-14 12:18:27 +02:00
vikke1234
50f93d14ff patterns/fdt: Fix flattened device tree parsing (#140)
* Fix flattened device tree parsing

Fix parsing to support multiple FDTs in a file and
non-zero base addresses.

* Fix string parsing

Parent offset wasn't being taken into account
when looking for prop names.
2023-07-13 14:38:52 +02:00
Nik
ab43516517 includes/std: Allow std::mem::MagicSearch to work with non-zero base addresses 2023-07-13 12:10:54 +02:00
Nik
4587b465d4 includes/std: Fixed std::mem::eof() not working correctly with non-zero base addresses 2023-07-13 12:08:25 +02:00
Nik
75b2c7be7e includes/std: Fixed MagicSearch type 2023-07-13 12:06:02 +02:00
Ange Albertini
19dd39e7c0 patterns: Added SE Linux binary format pattern (#139)
* SE Linux policy binary

module, base, kernel and packages

* Updated README with SELinux patterns

* SELinux test files
2023-07-12 20:25:47 +02:00
Nik
ecd34d35b2 patterns/elf: Fixed p_data definition in Phdr structs 2023-07-12 14:10:01 +02:00
Nik
06366839aa includes/std: Added std::ByteSizedArray 2023-07-12 00:13:02 +02:00
Nik
e3c387d0cf patterns/3ds: Added 3DS Max pattern 2023-07-10 16:36:49 +02:00
brliron
5a7077412c patterns/bmp: Various improvements (#138)
- Add various version of the BitmapInfoHeader struct, and autodetect
  the correct one.
- Fix palette detection.
- Use BitmapInfoHeader::biSize to find the position of the palette / data.
2023-07-10 11:31:07 +02:00
WerWolv
d8a37b0579 tests: Added WAD test file 2023-07-09 10:05:47 +02:00
WerWolv
1f1b2b9c5b tests: Fixed name of afe2 test file 2023-07-09 10:05:40 +02:00
Nik
d1645d2dc2 patterns:/intel_hex: Allow Intel Hex pattern to support files with linux line endings 2023-07-09 00:39:23 +02:00
WerWolv
61ef68b12a tests: Added more test files 2023-07-09 00:31:48 +02:00
WerWolv
c52e5b959b patterns: Fixed intel hex pattern 2023-07-09 00:14:58 +02:00
WerWolv
4a62892411 tests: Added various new test files 2023-07-09 00:14:49 +02:00
Nemoumbra
fc9b4cdbc2 tests: Added ico test file (#137) 2023-07-08 23:33:20 +02:00
Nik
daa556db31 tests: Fixed name of jpeg test pattern 2023-07-08 23:15:42 +02:00
Nik
e01a832fab patterns/lua54: Fixed parameter named const 2023-07-08 23:07:05 +02:00
Nik
759708d446 patterns/png: Automatically name PNG chunks 2023-07-03 22:06:48 +02:00
Trevor Gross
82560e6d9d patterns: Add a pattern for zlib compression (#135)
Add a pattern for RFC 1950 zlib compression
2023-07-02 18:58:19 +02:00
Nik
1a2d785093 patterns/java_class: Added MIME type 2023-06-24 11:05:10 +02:00
Nik
addec74d91 type/leb128: Fixed type always displaying 0 2023-06-22 17:35:59 +02:00
jfsbrito
ce83eedf02 patterns/7z: Added CompressData length and other improvements (#134)
* Update 7z.hexpat - v1.01

v1.01
--Minor changes.
--CompressData lenght added
--Output optimization

* Update 7z.hexpat v1.02

-Minor Changes
-Added Bzip2 verification
2023-06-22 07:27:31 +02:00
Doriel Rivalet
463ddcc62e themes: Add Catppuccin themes (#132)
* docs(README): add catppuccin theme

* themes: add catppuccin flavors

* chore: rename catppuccin files

* docs(README): update catppuccin file names
2023-06-19 23:09:16 +02:00
jfsbrito
13b97fc976 patterns/7z: Added 7Zip pattern (#128)
* 7z.hexpat

This repository contains a custom pattern for the ImHex tool that enables the analysis of 7zip files. The pattern allows for a structured view of 7zip files within ImHex, providing insights into various aspects of the file structure.

- Identification of 7zip files: The custom pattern detects and identifies 7zip file types based on their signature.
- Extraction of key information: The pattern extracts important details from the 7zip file, such as the format version, CRC values, relative offset of the end header, and file size.
- Differentiation of compression methods: The pattern distinguishes between LZMA and LZMA2 compression methods used within the 7zip file.
- Visualization of start and end headers: The pattern helps in visualizing the start and end headers of the 7zip file, making it easier to navigate through its structure.

To use the custom pattern for analyzing 7zip files in ImHex, follow these steps:

1. Install ImHex: Ensure that you have ImHex installed on your system.
2. Open a 7zip file: Launch ImHex and open the 7zip file you want to analyze.
3. Apply the custom pattern: In the "Pattern" menu, select the custom pattern for 7zip file analysis and apply it.
4. Explore the file structure: ImHex will display the matched patterns, allowing you to explore the structure of the 7zip file with the extracted information.

Please note that this custom pattern is designed specifically for 7zip files and may not be suitable for other file formats.

Contributions to this custom pattern for 7zip file analysis are welcome. If you have any improvements, bug fixes, or suggestions, feel free to submit a pull request.

This custom pattern for ImHex is licensed under the [MIT License](LICENSE.md). You are free to modify and distribute this pattern as per the terms of the license.

We would like to acknowledge the developers of ImHex for providing an excellent tool for binary file analysis. The custom pattern for 7zip file analysis in this repository builds upon the capabilities of ImHex to enhance the understanding of 7zip file structures.

---

We hope you find this custom pattern useful for analyzing 7zip files using ImHex. If you encounter any issues or have any questions, please feel free to raise them in the "Issues" section of this repository.

Thanks!

* Added test file

---------
2023-06-19 23:08:42 +02:00
luisfilipe23lol
55e4283432 patterns/cda: Added CDA pattern (#133)
* Create cda.hexpat

pattern language created for .cda files

* added test file for cda pattern

* Delete cda.hexpat.cda.cda

* Add test file cda

test file for cda pattern

* Update README.md

Add cda file in read.me
2023-06-19 14:22:17 +02:00
Nik
ea225edf12 git: Disable coverage information for now 2023-06-17 15:20:01 +02:00
blondecake433
bb19cb43ee patterns:/mp4: Added mp4 pattern (#129) 2023-06-17 15:07:08 +02:00
Nik
8f8ad0e2d5 git: Try fix Coveralls heap out of memory 2023-06-17 15:02:53 +02:00
Nik
3ad1f3969f includes/std: Fixed documentation of copy_value_to_section function 2023-06-15 22:58:44 +02:00
cryptax
5451d45158 patterns/dex: Fix pattern to see data section of DEX files + Link_data (#127) 2023-06-15 18:48:47 +02:00
Nik
e6a731fa1d includes/type: Added Base64 type 2023-06-15 17:34:20 +02:00
Jackson Barreto
8a62001705 patterns: Added TIFF pattern (#126)
* Create tiff.hexpat

* Update README.md

Update readme to include tiff hexpat

* added file to test the pattern

---------

Co-authored-by: joelalves <joel.17.alves@gmail.com>
2023-06-15 08:08:39 +02:00
Nik
1f8710b586 patterns/ntag: Fixed Length type transform function 2023-06-13 13:36:53 +02:00
Nik
acd2d4abb8 includes/std: Fixed description of array library (#125) 2023-06-12 08:10:48 +02:00
qufb
032f3c7c01 pattern/midi: Support multiple tracks (#124)
* pattern/midi: Support multiple tracks

* pattern/midi: Replace custom type used to index array
2023-06-11 13:09:23 +02:00
无以铭川
3841ff51ef pattern/java_class: Format byte code instruction (#123) 2023-06-08 23:39:39 +02:00
Nik
622721403f patterns/flac: Fixed typo 2023-06-06 08:26:01 +02:00
Nik
11291d1ebb tests: Added JPG test file 2023-06-05 10:59:12 +02:00
Nik
92e61ccf5d tests: Fixed build with new pattern language changes 2023-06-05 10:03:55 +02:00
Nik
062edfe527 includes/std: Fixed std::fxpt::to_fixed conversion issue 2023-06-05 09:48:33 +02:00
paxcut
5b32941801 patterns: Added visualizers to image patterns (#117)
* Added image visualizers to image patterns that were supported

* missing include files

* Small style fixes

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2023-06-04 22:38:25 +02:00
Nik
e99ab5b59b patterns/java_class: Fixed duplicate variables 2023-06-02 10:54:09 +02:00
Feitong Chen
775c836766 patterns/macho: Fix macho CpuType error and add CpuType::ARM64 and CpuType::ARM64_32 (#115) 2023-06-01 16:38:34 +02:00
blondecake433
43058b4c45 pattern/gif: Improved GIF pattern (#114) 2023-05-26 09:28:18 +02:00
Justus Garbe
0128ea87db patterns/java_class: Added Java SE 20 Attributes, added value displaying (#113)
greatly enchanced the java pattern
2023-05-19 17:59:15 +02:00
WerWolv
fe231436d9 patterns/gb: Added Gameboy ROM pattern (#89)
* Added gameboy format

* Add mime type

* Add test

* Avoid DMCA :sweating:
2023-05-19 16:09:14 +02:00
gmestanley
7a9ac44577 patterns/pe: Improved PE pattern (#112)
New improvement
2023-05-16 11:03:07 +02:00
Nick Bitounis
74e9d8f2e7 patterns: Added pattern file for M.A.X. v1.04 game files. (#111)
* Added pattern file for M.A.X. v1.04 game files.

* * Added MAX pattern in list of patterns.
* Added MAX sample game save file.
* Reordered the list of patterns to ascending name.

---------

Co-authored-by: n.bitounis <n.bitounis@xe.gr>
2023-05-08 16:17:19 +02:00
Nik
2f39a62d85 includes/std: Added support for writing patterns to files 2023-05-02 20:38:21 +02:00
AdventureT
e21063d58b patterns: Added Crash Bandicoot - Back in time level format (#110)
* Added Crash Bandicoot - Back in time level format

* Update README.md

* Fix README.md
2023-04-25 21:44:02 +02:00
paxcut
8f1a6bdd75 includes/std: Added std::math::exp(), fixed std::math::ln comment (#106)
Removed extraneous argument base to ln function and added the necessary definition in include file to access previously builtin function exp.
2023-04-17 22:19:47 +02:00
Nik
b9af43e08b patterns: Fixed WAV MIME type in readme 2023-04-16 21:37:40 +02:00
Nik
2ffa4e0161 patterns/wav: Fixed MIME type 2023-04-06 12:33:46 +02:00
Zaggy1024
1cd7f92a5d patterns/includes: Update standard library and patterns to support the new bitfields (#102)
* Add `current_bit_offset()` and `read_bits(...)` to `std::mem`

* Replace deprecated BitfieldOrder enum values with new clearer names

This adds new options named `MostToLeastSignificant` and `LeastToMostSignificant` to replace the old `LeftToRight` and `RightToLeft` names. These names should be much clearer about what they affect and how.

* Throw errors when `std::core::(get|set)_bitfield_order()` are called

* Update all patterns to work with the new bitfield behaviors
2023-04-01 11:16:54 +02:00
Nik
d42b87d9e6 git: Added CONTRIBUTING.md 2023-03-29 08:10:58 +02:00
Nik
035de359d7 git: Updated readme information 2023-03-29 08:03:40 +02:00
Nik
fe59788783 includes/type: Fixed IP library top level comment 2023-03-28 18:39:46 +02:00
Nik
ef198cf24f includes/type: Fixed Path formatter 2023-03-28 15:53:19 +02:00
Nik
b73b69a8cc includes: Added documentations for all remaining types and functions 2023-03-28 15:28:44 +02:00
Nik
5d72494019 includes/hex: Added ImHex check to provider library 2023-03-28 10:21:03 +02:00
Nik
afbce642fb includes/hex: Added provider functions 2023-03-28 09:49:12 +02:00
Nik
a72058eb65 includes/std: Added std::random functions 2023-03-28 09:28:28 +02:00
Chris Bailey
ab2ed98dab includes/std: Added nullable pointers to std/ptr (#97) 2023-03-27 10:05:12 +02:00
WerWolv
3edc6ea172 includes: Added documentation for a few more files 2023-03-26 16:58:39 +02:00
WerWolv
a9a7f0b186 includes: Added documentation to the rest of the std and hex library 2023-03-26 16:34:06 +02:00
Nik
5f352e26d1 patterns/elf: Make displaying section names optional 2023-03-26 11:34:52 +02:00
Nik
3e2a6aabaa patterns/evtx: Fixed type name typo 2023-03-26 11:22:15 +02:00
Nik
56d6f0187f patterns/zstd: Fixed block_header_t bitfield 2023-03-26 11:09:26 +02:00
Vladimir
adf7256c39 patterns/bsp: Added GoldSrc engine maps file format (#101)
Co-authored-by: Nik <werwolv98@gmail.com>
2023-03-26 10:36:03 +02:00
dora
a25a8a3615 patterns/evtx: Added evtx pattern (#100)
* add evtx pattern

* fix Readme

* fix coding style

* space adjustment

* space adjustment
2023-03-26 10:34:45 +02:00
WerWolv
6ae8b30488 git: Allow documentation to be regenerated when PatternLanguage changes were made 2023-03-26 10:26:40 +02:00
Nik
125ba38d72 plugins/extra-hashes: Fixed ImHex version 2023-03-25 15:33:55 +01:00
Nik
ba55feb200 plugins: Added Extra Hashes plugin to database 2023-03-25 14:47:09 +01:00
Nik
a301a4bfeb themes: Added Solarized Dark theme 2023-03-23 17:20:05 +01:00
Nik
531be04739 themes/vs_dark: Fixed window title alignment 2023-03-23 14:38:39 +01:00
Nik
87e05bec48 git: Fixed uefi pattern link in readme 2023-03-23 11:59:38 +01:00
Morten Linderud
a31d290005 patterns/uefi: Added UEFI pattern (#8)
Added UEFI structs

Signed-off-by: Morten Linderud <morten@linderud.pw>
Co-authored-by: Nik <werwolv98@gmail.com>
2023-03-23 11:52:50 +01:00
Andrei Makeev
f3de35a320 patterns/elf: Make hex view less ambiguous (#92)
* ELF: make section/segment data arrays sealed

* ELF: set display names for section/segment data

* Added new functions to library

* Updated ELF pattern

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2023-03-23 11:51:01 +01:00
Oded Shapira
d9de2f7058 patterns/lua54: Added new Lua 5.4 bytecode pattern (#80)
* Lua 5.4 pattern

* Fixed formatting, added file to readme

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2023-03-23 09:06:24 +01:00
Jozef Izso
790f837e4f patterns/dsstore: Added .DS_Store pattern (#90)
* Initial format for Buddy Allocator structures in `.DS_Store` files

* Add list of entries for files in `.DS_Store`

* Add root block, offsets, toc and free lists structures

* Add parsing of block data

* Document `.DS_Store` pattern and add test file
2023-03-23 08:59:59 +01:00
Josh
d8a291977a encodings: Added Pokemon gen 3 character encoding (#96)
* added Pokemon Emerald character encoding

* update readme.md for generic gen 3 table

* Update and rename pokeEmerald_en.tbl to pokegen3_en.tbl
2023-03-23 08:58:44 +01:00
Nicolás Alvarez
62a83b53aa patterns/pbz: Modernize code, renamed pattern to pbz (#98)
* Update pbz pattern

- Rename pbzx to pbz; turns out the 'x' is the compression type.
- Use type::Magic for the "pbz" magic number.
- Decode compression type as an enum.
- Mention compression_tool in the header comment.

* Rename pbzx.hexpat to pbz.hexpat
2023-03-23 08:57:33 +01:00
Takumi Sueda
53ea45ffa6 patterns/PCAP: Fixed formatting and added endianess support (#99)
* patterns/pcap: reformat

* patterns/pcap: endianness-aware parse / parse packets until EOF
2023-03-23 08:56:20 +01:00
WerWolv
c0a1bbd218 includes/std: Fixed stray " in bit library 2023-03-19 16:30:54 +01:00
WerWolv
27f4e20638 git: Fixed documentation dispatch 2023-03-19 16:26:21 +01:00
WerWolv
bbb2107d5f git: Always execute dispatch 2023-03-19 16:21:46 +01:00
WerWolv
8601a6665e includes/std: Added first set of documentation to the std library 2023-03-19 16:17:04 +01:00
WerWolv
c6f2b57384 git: Fixed workflow name 2023-03-19 11:23:43 +01:00
WerWolv
dd190f7c8a git: Added documentation dispatch workflow 2023-03-19 11:22:55 +01:00
Nik
55e3fec3bc includes/std: Added std::math::accumulate, corrected std::hash::crc32 2023-03-14 14:40:58 +01:00
WerWolv
8e78f371f5 encodings/utf8: Corrected UTF-8 encoding file 2023-03-14 09:36:59 +01:00
Ahmet Bilal Can
2758ec8d36 patterns/dex: Added more information to display (#95)
Added string representations for:
- string ids
- field ids
- method ids
- class ids
- proto ids
2023-03-13 11:43:43 +01:00
WerWolv
82ca79c166 patterns/usb: Fixed duplicate variable declaration 2023-03-13 11:35:09 +01:00
WerWolv
146273b1b3 tests: Fixed compilation with latest libpl 2023-03-13 11:19:20 +01:00
Andrei Makeev
ab4bff9f42 patterns/elf: Treat SHT_SYMTAB as symbol tables (#91)
ELF files: support static symbol tables
2023-03-10 11:03:36 +01:00
cryptax
000f0eb730 patterns/dex: Adding Dalvik Executable pattern (#94)
* adding DEX file format

* adding map list
2023-03-10 11:03:07 +01:00
qux-bbb
2d45d5d086 patterns/pe: Removed unnecessary offset (#87)
Remove unnecessary offsets
2023-02-22 21:05:13 +01:00
Nik
2d4b4add82 includes/std: Added Bytes type 2023-02-22 18:31:52 +01:00
Nik
e635c3a5bf patterns/stl: Added 3D model visualization 2023-02-20 11:35:15 +01:00
Chris
15234a284d includes/type: Add signed LEB128 type support (#86) 2023-02-17 20:35:27 +01:00
Nik
0673673b99 themes/vs_dark: Added new Visual Studio Dark theme 2023-02-16 18:11:05 +01:00
Justus Garbe
3786f7e265 patterns/gif: Improved gif format using more explicit formating styles (#84)
* Improved gif format using more explicit formating styles

* Set mime type

* Move content type formatting to struct value
2023-02-15 09:40:09 +01:00
Kuruyia
ea4dda001a includes/type: Fix the IPv6 type (#83)
includes/type: fix the IPv6 type
2023-02-13 08:49:29 +01:00
254 changed files with 190833 additions and 1120812 deletions

2
.gitattributes vendored
View File

@@ -1,2 +1,4 @@
*.pat linguist-language=Rust
*.hexpat linguist-language=Rust
tests/patterns/test_data/** binary

View File

@@ -5,6 +5,7 @@
## Checklist
- [ ] A pattern for this format doesn't exist yet (or this PR improves the existing one)
- [ ] The new pattern has been added to the relevant table in the Readme
- [ ] The new pattern has a description pragma (`#pragma description My pattern Description here`)
- [ ] The pattern was associated with all relevant MIME types (using `#pragma MIME mime-type` in the source code)
- Make sure to never use `application/octet-stream` here as that means "Unidentifiable binary data"
- [ ] A test file for this pattern has been added to [/tests/patterns/test_data](/tests/patterns/test_data)

56
.github/workflows/dispatch.yml vendored Normal file
View File

@@ -0,0 +1,56 @@
name: "Dispatch"
on:
push:
branches: [ '*' ]
pull_request:
branches: [ '*' ]
repository_dispatch:
types: [run_tests]
workflow_dispatch:
inputs:
generate_docs:
description: "Regenerate docs"
required: false
type: boolean
jobs:
tests:
name: 🎯 Dispatch changes
runs-on: ubuntu-22.04
env:
DISPATCH_TOKEN: ${{ secrets.DISPATCH_TOKEN }}
permissions:
actions: read
contents: read
security-events: write
steps:
- name: 🧰 Checkout
uses: actions/checkout@v2
with:
submodules: recursive
- name: 📄 Check changed include files
id: changed-includes
uses: tj-actions/changed-files@v35
with:
files: includes/**/*.pat
- name: ✉️ Run Documentation generator
if: ${{ env.DISPATCH_TOKEN != '' && (steps.changed-includes.outputs.any_changed == 'true' || inputs.generate_docs || github.event_name == 'repository_dispatch' ) }}
uses: mvasigh/dispatch-action@main
with:
token: ${{ secrets.DISPATCH_TOKEN }}
repo: Documentation
owner: WerWolv
event_type: update_pl_docs
- name: ✉️ Update PatternLanguage Website
if: ${{ env.DISPATCH_TOKEN != '' }}
uses: mvasigh/dispatch-action@main
with:
token: ${{ secrets.DISPATCH_TOKEN }}
repo: PatternLanguageWeb
owner: WerWolv
event_type: rebuild_wasm

View File

@@ -28,6 +28,7 @@ jobs:
sudo apt update
sudo apt install -y \
build-essential \
ccache \
gcc-12 \
g++-12 \
lld \
@@ -41,12 +42,21 @@ jobs:
sudo pip install jsonschema
- name: 📜 Setup ccache
uses: hendrikmuhs/ccache-action@v1
with:
key: ${{ runner.os }}-ccache-${{ github.run_id }}
restore-keys: ${{ runner.os }}-ccache
max-size: 1G
- name: 🛠️ Build
run: |
cd tests
mkdir -p build
cd build
CC=gcc-12 CXX=g++-12 cmake \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_FLAGS="-fuse-ld=lld --coverage" \
-DCMAKE_CXX_FLAGS="-fuse-ld=lld --coverage" \
-DLIBPL_ENABLE_TESTS=OFF \
@@ -57,18 +67,7 @@ jobs:
- name: 🧪 Perform Unit Tests
run: |
cd tests/build
ctest --output-on-failure
- name: ⚗️ Generate Coverage Report
run: |
cd tests/build
lcov -d ./_deps/pattern_language-build/lib --gcov-tool /usr/bin/gcov-12 -c -o coverage.info
- name: ⬆️ Upload Coverage Report
uses: coverallsapp/github-action@master
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: tests/build/coverage.info
ctest --output-on-failure -j 4
- name: 📎 Validate JSON Files
run: |

4
.gitignore vendored
View File

@@ -1,4 +1,6 @@
tests/cmake-build-debug/
tests/cmake*/
tests/build*/
.idea/
.DS_Store

8
.gitmodules vendored
View File

@@ -1,4 +1,10 @@
[submodule "rules"]
path = yara/official_rules
url = https://github.com/Yara-Rules/rules
branch = master
branch = master
[submodule "patterns/ffx"]
path = patterns/ffx
url = https://gitlab.com/EvelynTSMG/imhex-ffx-pats.git
[submodule "patterns/bastion"]
path = patterns/bastion
url = https://gitlab.com/EvelynTSMG/imhex-bastion-pats.git

12
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,12 @@
# Contributing
Contributing to the Database is very simple. Fork this repository, add your new files (or changes to existing files) to the right folders and create a PR.
When adding new files, please make sure to also add a link to them to the Table of Contents in the README.md file. That way the files can be discovered by others more easily.
Thanks a lot for any additions or improvements :)
## Adding new Patterns
When adding new patterns, if possible, please also add a test file named `<pattern_name>.hexpat.<extension>` to the `/tests/patterns/test_data` directory. This allows our Unit Tests to be run against your code so we can make sure it stays up-to-date and doesn't break when changes are made to the PatternLanguage.
Please try to keep these files as small as possible (~100kiB at most) so cloning stays fast.
Please also make sure to not submit any test files that are under copyright such as game files, ROMs or files extracted from other programs. We don't want a DMCA takedown on this repo.

203
README.md
View File

@@ -1,6 +1,21 @@
# ImHex-Patterns
# ImHex Database
Hex patterns, include patterns and magic files for the use with the ImHex Hex Editor
This repository serves as a database for files to use with the [ImHex Hex Editor](https://github.com/WerWolv/ImHex). It currently contains
- [Patterns](/patterns) - Binary Format definitions for the Pattern Language
- [Pattern Libraries](/includes) - Libraries that make using the Pattern Language easier
- [Magic Files](/magic) - Custom magic file definitions for the use with libmagic
- [Encodings](/encodings) - Custom encodings in the .tbl format
- [Data Processor Nodes](/nodes) - Custom nodes made for ImHex's Data Processor
- [Themes](/themes) - Custom themes for ImHex
- [Constants](/constants) - Constants definition files
- [Scripts](/scripts) - Various scripts to generate code or automate some tasks
- [Yara](/yara) - Custom Yara rules
## Submissions
Most files in this repository have been submitted by the community. Please feel free to open a PR on your own and add files to it!
Everything will immediately show up in ImHex's Content Store and gets bundled with the next release of ImHex.
## Table of Contents
@@ -8,69 +23,115 @@ Hex patterns, include patterns and magic files for the use with the ImHex Hex Ed
| Name | MIME | Path | Description |
|------|------|------|-------------|
| 3DS | | [`patterns/3ds.hexpat`](patterns/3ds.hexpat) | Autodesk 3DS Max Model file |
| 7Z | | [`patterns/7z.hexpat`](patterns/7z.hexpat) | 7z File Format |
| AFE2 | | [`patterns/afe2.hexpat`](patterns/afe2.hexpat) | Nintendo Switch Atmosphère CFW Fatal Error log |
| ANI | `application/x-navi-animation` | [`patterns/ani.hexpat`](patterns/ani.hexpat) | Windows Animated Cursor file |
| AR | `application/x-archive` | [`patterns/ar.hexpat`](patterns/ar.hexpat) | Static library archive files |
| ARIA2 | | [`patterns/aria2.hexpat`](patterns/aria2.hexpat) | ARIA2 Download Manager Control files |
| ARM VTOR | | [`patterns/arm_cm_vtor.hexpat`](patterns/arm_cm_vtor.hexpat) | ARM Cortex M Vector Table Layout |
| Bastion | | [`patterns/bastion/*`](https://gitlab.com/EvelynTSMG/imhex-bastion-pats) | Various [Bastion](https://en.wikipedia.org/wiki/Bastion_(video_game)) files |
| Bencode | `application/x-bittorrent` | [`patterns/bencode.hexpat`](patterns/bencode.hexpat) | Bencode encoding, used by Torrent files |
| BMP | `image/bmp` | [`patterns/bmp.hexpat`](patterns/bmp.hexpat) | OS2/Windows Bitmap files |
| ELF | `application/x-executable` | [`patterns/elf.hexpat`](patterns/elf.hexpat) | ELF header in elf binaries |
| PE | `application/x-dosexec` | [`patterns/pe.hexpat`](patterns/pe.hexpat) | PE header, COFF header, Standard COFF fields and Windows Specific fields |
| NE | | [`patterns/ne.hexpat`](patterns/ne.hexpat) | NE header and Standard NE fields |
| Intel HEX | | [`patterns/intel_hex.hexpat`](patterns/intel_hex.hexpat) | [Intel hexadecimal object file format definition]("https://en.wikipedia.org/wiki/Intel_HEX") |
| MIDI | `audio/midi` | [`patterns/midi.hexpat`](patterns/midi.hexpat) | MIDI header, event fields provided |
| WAV | `audio/wav` | [`patterns/wav.hexpat`](patterns/wav.hexpat) | RIFF header, WAVE header, PCM header |
| ZIP | `application/zip` | [`patterns/zip.hexpat`](patterns/zip.hexpat) | End of Central Directory Header, Central Directory File Headers |
| PCAP | `application/vnd.tcpdump.pcap` | [`patterns/pcap.hexpat`](patterns/pcap.hexpat) | pcap header and packets |
| SPIRV | | [`patterns/spirv.hexpat`](patterns/spirv.hexpat) | SPIR-V header and instructions |
| AFE2 | | [`patterns/afe2.hexpat`](patterns/afe2.hexpat) | Nintendo Switch Atmosphère CFW Fatal Error log |
| AR | `application/x-archive` | [`patterns/ar.hexpat`](patterns/ar.hexpat) | Static library archive files |
| NACP | | [`patterns/nacp.hexpat`](patterns/nacp.hexpat) | Nintendo Switch NACP files |
| NRO | | [`patterns/nro.hexpat`](patterns/nro.hexpat) | Nintendo Switch NRO files |
| PRODINFO | | [`patterns/prodinfo.hexpat`](patterns/prodinfo.hexpat) | Nintendo Switch PRODINFO |
| Java Class | `application/x-java-applet` | [`patterns/java_class.hexpat`](patterns/java_class.hexpat) | Java Class files |
| ARM VTOR | | [`patterns/arm_cm_vtor.hexpat`](patterns/arm_cm_vtor.hexpat) | ARM Cortex M Vector Table Layout |
| ICO | | [`patterns/ico.hexpat`](patterns/ico.hexpat) | Icon (.ico) or Cursor (.cur) files |
| PNG | `image/png` | [`patterns/png.hexpat`](patterns/png.hexpat) | PNG image files |
| QOI | `image/qoi` | [`patterns/qoi.hexpat`](patterns/qoi.hexpat) | QOI image files |
| DDS | `image/vnd-ms.dds` | [`patterns/dds.hexpat`](patterns/dds.hexpat) | DirectDraw Surface |
| TGA | `image/tga` | [`patterns/tga.hexpat`](patterns/tga.hexpat) | Truevision TGA/TARGA image |
| ISO | | [`patterns/iso.hexpat`](patterns/iso.hexpat) | ISO 9660 file system |
| VDF | | [`patterns/vdf.hexpat`](patterns/vdf.hexpat) | Binary Value Data Format (.vdf) files |
| IP | | [`patterns/ip.hexpat`](patterns/ip.hexpat) | Ethernet II Frames (IP Packets) |
| UF2 | | [`patterns/uf2.hexpat`](patterns/uf2.hexpat) | [USB Flashing Format](https://github.com/microsoft/uf2) |
| File System | | [`patterns/fs.hexpat`](patterns/fs.hexpat) | Drive File System |
| Bencode | `application/x-bittorrent` | [`patterns/bencode.hexpat`](patterns/bencode.hexpat) | Bencode encoding, used by Torrent files |
| Protobuf | | [`patterns/protobuf.hexpat`](patterns/protobuf.hexpat) | Google Protobuf encoding |
| OGG | `audio/ogg` | [`patterns/ogg.hexpat`](patterns/ogg.hexpat) | OGG Audio format |
| STL | `model/stl` | [`patterns/stl.hexpat`](patterns/stl.hexpat) | STL 3D Model format |
| VHDX | | [`patterns/vhdx.hexpat`](patterns/vhdx.hexpat) | Microsoft Hyper-V Virtual Hard Disk format |
| NTAG | | [`patterns/ntag.hexpat`](patterns/ntag.hexpat) | NTAG213/NTAG215/NTAG216, NFC Forum Type 2 Tag compliant IC |
| Shell Link | `application/x-ms-shortcut` | [`patterns/lnk.hexpat`](patterns/lnk.hexpat) | Windows Shell Link file format |
| Xilinx BIT | | [`patterns/xilinx_bit.hexpat`](patterns/xilinx_bit.hexpat) | Xilinx FPGA Bitstreams |
| FLAC | `audio/flac` | [`patterns/flac.hexpat`](patterns/flac.hexpat) | Free Lossless Audio Codec, FLAC Audio Format |
| BSON | `application/bson` | [`patterns/bson.hexpat`](patterns/bson.hexpat) | BSON (Binary JSON) format |
| msgpack | `application/x-msgpack` | [`patterns/msgpack.hexpat`](patterns/msgpack.hexpat) | MessagePack binary serialization format |
| MiniDump | `application/x-dmp` | [`patterns/minidump.hexpat`](patterns/minidump.hexpat) | Windows MiniDump files |
| ID3 | `audio/mpeg` | [`patterns/id3.hexpat`](patterns/id3.hexpat) | ID3 tags in MP3 files |
| TAR | `application/x-tar` | [`patterns/tar.hexpat`](patterns/tar.hexpat) | Tar file format |
| CPIO | `application/x-cpio` | [`patterns/cpio.hexpat`](patterns/cpio.hexpat) | Old Binary CPIO Format |
| FDT | | [`patterns/fdt.hexpat`](patterns/fdt.hexpat) | Flat Linux Device Tree blob |
| StuffItV5 | `application/x-stuffit` | [`patterns/sit5.hexpat`](patterns/sit5.hexpat) | StuffIt V5 archive |
| NBT | | [`patterns/nbt.hexpat`](patterns/nbt.hexpat) | Minecraft NBT format |
| PCX | `application/x-pcx` | [`patterns/pcx.hexpat`](patterns/pcx.hexpat) | PCX Image format |
| GZIP | `application/gzip` | [`patterns/gzip.hexpat`](patterns/gzip.hexpat) | GZip compressed data format |
| PFS0 | | [`patterns/pfs0.hexpat`](patterns/pfs0.hexpat) | Nintendo Switch PFS0 archive (NSP files) |
| XCI | | [`patterns/xci.hexpat`](patterns/xci.hexpat) | Nintendo Switch XCI cardridge ROM |
| WAD | | [`patterns/wad.hexpat`](patterns/wad.hexpat) | DOOM WAD Archive |
| GIF | `image/gif` | [`patterns/gif.hexpat`](patterns/gif.hexpat) | GIF image files |
| ZSTD | `application/zstd` | [`patterns/zstd.hexpat`](patterns/zstd.hexpat) | Zstandard compressed data format |
| COFF | `application/x-coff` | [`patterns/coff.hexpat`](patterns/coff.hexpat) | Common Object File Format (COFF) executable |
| Mach-O | `application/x-mach-binary` | [`patterns/macho.hexpat`](patterns/macho.hexpat) | Mach-O executable |
| CHM | | [`patterns/chm.hexpat`](patterns/chm.hexpat) | Windows HtmlHelp Data (ITSF / CHM) |
| DMG | | [`patterns/dmg.hexpat`](patterns/dmg.hexpat) | Apple Disk Image Trailer (DMG) |
| XBEH | `audio/x-xbox-executable` | [`patterns/xbeh.hexpat`](patterns/xbeh.hexpat) | Xbox executable |
| QBCL | | [`patterns/qbcl.hexpat`](patterns/qbcl.hexpat) | Qubicle voxel scene project file |
| BIN | | [`patterns/selinux.hexpat`](patterns/selinux.pat) | SE Linux modules |
| BSON | `application/bson` | [`patterns/bson.hexpat`](patterns/bson.hexpat) | BSON (Binary JSON) format |
| bplist | | [`patterns/bplist.hexpat`](patterns/bplist.hexpat) | Apple's binary property list format (bplist) |
| BSP | | [`patterns/bsp_goldsrc.hexpat`](patterns/bsp_goldsrc.hexpat) | GoldSrc engine maps format (used in Half-Life 1) |
| CCHVA | | [`patterns/cchva.hexpat`](patterns/cchva.hexpat) | Command and Conquer Voxel Animation |
| CCVXL | | [`patterns/ccvxl.hexpat`](patterns/ccvxl.hexpat) | Command and Conquer Voxel Model |
| CCPAL | | [`patterns/ccpal.hexpat`](patterns/ccpal.hexpat) | Command and Conquer Voxel Palette |
| PIF | `image/pif` | [`patterns/pif.hexpat`](patterns/pif.hexpat) | PIF Image Format |
| CDA | | [`patterns/cda.hexpat`](patterns/cda.hexpat) | Compact Disc Audio track |
| CHM | | [`patterns/chm.hexpat`](patterns/chm.hexpat) | Windows HtmlHelp Data (ITSF / CHM) |
| COFF | `application/x-coff` | [`patterns/coff.hexpat`](patterns/coff.hexpat) | Common Object File Format (COFF) executable |
| CPIO | `application/x-cpio` | [`patterns/cpio.hexpat`](patterns/cpio.hexpat) | Old Binary CPIO Format |
| CrashLvl | | [`patterns/Crashlvl.hexpat`](patterns/Crashlvl.hexpat) | Crash Bandicoot - Back in Time (fan game) User created level format |
| DDS | `image/vnd-ms.dds` | [`patterns/dds.hexpat`](patterns/dds.hexpat) | DirectDraw Surface |
| DEX | | [`patterns/dex.hexpat`](patterns/dex.hexpat) | Dalvik EXecutable Format |
| DMG | | [`patterns/dmg.hexpat`](patterns/dmg.hexpat) | Apple Disk Image Trailer (DMG) |
| DS_Store | | [`patterns/dsstore.hexpat`](patterns/dsstore.hexpat) | .DS_Store file format |
| DTA | | [`patterns/max_v104.hexpat`](patterns/max_v104.hexpat) | Mechanized Assault and Exploration v1.04 (strategy game) save file format |
| DTED | | [`patterns/dted.hexpat`](patterns/dted.hexpat) | Digital Terrain Elevation Data (DTED) |
| ELF | `application/x-executable` | [`patterns/elf.hexpat`](patterns/elf.hexpat) | ELF header in elf binaries |
| EVTX | | [`patterns/evtx.hexpat`](patterns/evtx.hexpat) | MS Windows Vista Event Log |
| FAS | | [`patterns/fas_oskasoftware.hexpat`](patterns/fas_oskasoftware.hexpat) [`patterns/fas_oskasoftware_old.hexpat`](patterns/fas_oskasoftware_old.hexpat) (Old versions of Oska DeskMate) | Oska Software DeskMates FAS (Frames and Sequences) file |
| FDT | | [`patterns/fdt.hexpat`](patterns/fdt.hexpat) | Flat Linux Device Tree blob |
| FFX | | [`patterns/ffx/*`](https://gitlab.com/EvelynTSMG/imhex-ffx-pats) | Various Final Fantasy X files |
| File System | | [`patterns/fs.hexpat`](patterns/fs.hexpat) | Drive File System |
| FLAC | `audio/flac` | [`patterns/flac.hexpat`](patterns/flac.hexpat) | Free Lossless Audio Codec, FLAC Audio Format |
| GB | `application/x-gameboy-rom` | [`patterns/gb.hexpat`](patterns/gb.hexpat) | Gameboy ROM |
| GGUF | | [`patterns/gguf.hexpat`](patterns/gguf.hexpat) | GGML Inference Models |
| GIF | `image/gif` | [`patterns/gif.hexpat`](patterns/gif.hexpat) | GIF image files |
| GZIP | `application/gzip` | [`patterns/gzip.hexpat`](patterns/gzip.hexpat) | GZip compressed data format |
| Halo Tag || [`patterns/hinf_tag.hexpat`](patterns/hinf_tag.hexpat) | Halo Infinite Tag Files |
| Halo Module || [`patterns/hinf_module.hexpat`](patterns/hinf_module.hexpat) | Halo Infinite Module Archive Files |
| Halo HavokScript || [`patterns/hinf_luas.hexpat`](patterns/hinf_luas.hexpat) | Halo Infinite HavokScript 5.1 Bytecode |
| ICO | | [`patterns/ico.hexpat`](patterns/ico.hexpat) | Icon (.ico) or Cursor (.cur) files |
| ID3 | `audio/mpeg` | [`patterns/id3.hexpat`](patterns/id3.hexpat) | ID3 tags in MP3 files |
| Intel HEX | | [`patterns/intel_hex.hexpat`](patterns/intel_hex.hexpat) | [Intel hexadecimal object file format definition]("https://en.wikipedia.org/wiki/Intel_HEX") |
| IP | | [`patterns/ip.hexpat`](patterns/ip.hexpat) | Ethernet II Frames (IP Packets) |
| IPS | | [`patterns/ips.hexpat`](patterns/ips.hexpat) | IPS (International Patching System) files |
| ISO | | [`patterns/iso.hexpat`](patterns/iso.hexpat) | ISO 9660 file system |
| Java Class | `application/x-java-applet` | [`patterns/java_class.hexpat`](patterns/java_class.hexpat) | Java Class files |
| JPEG | `image/jpeg` | [`patterns/jpeg.hexpat`](patterns/jpeg.hexpat) | JPEG Image Format |
| Lua 5.4 | | [`patterns/lua54.hexpat`](patterns/lua54.hexpat) | Lua 5.4 bytecode |
| Mach-O | `application/x-mach-binary` | [`patterns/macho.hexpat`](patterns/macho.hexpat) | Mach-O executable |
| MIDI | `audio/midi` | [`patterns/midi.hexpat`](patterns/midi.hexpat) | MIDI header, event fields provided |
| MiniDump | `application/x-dmp` | [`patterns/minidump.hexpat`](patterns/minidump.hexpat) | Windows MiniDump files |
| mp4 | `video/mp4` | [`patterns/mp4.hexpat`](patterns/mp4.hexpat) | MPEG-4 Part 14 digital multimedia container format |
| msgpack | `application/x-msgpack` | [`patterns/msgpack.hexpat`](patterns/msgpack.hexpat) | MessagePack binary serialization format |
| NACP | | [`patterns/nacp.hexpat`](patterns/nacp.hexpat) | Nintendo Switch NACP files |
| NBT | | [`patterns/nbt.hexpat`](patterns/nbt.hexpat) | Minecraft NBT format |
| NE | | [`patterns/ne.hexpat`](patterns/ne.hexpat) | NE header and Standard NE fields |
| nes | | [`patterns/nes.hexpat`](patterns/nes.hexpat) | .nes file format |
| NotepadWindowState | | [`patterns/notepad-windowstate.hexpat`](patterns/notepad-windowstate.hexpat) | Windows 11 Notepad - Window State .bin file |
| NRO | | [`patterns/nro.hexpat`](patterns/nro.hexpat) | Nintendo Switch NRO files |
| NTAG | | [`patterns/ntag.hexpat`](patterns/ntag.hexpat) | NTAG213/NTAG215/NTAG216, NFC Forum Type 2 Tag compliant IC |
| OGG | `audio/ogg` | [`patterns/ogg.hexpat`](patterns/ogg.hexpat) | OGG Audio format |
| PAK | | [`patterns/xgspak.hexpat`](patterns/xgspak.hexpat) | Exient XGS Engine Pak files |
| PCAP | `application/vnd.tcpdump.pcap` | [`patterns/pcap.hexpat`](patterns/pcap.hexpat) | pcap header and packets |
| PCX | `application/x-pcx` | [`patterns/pcx.hexpat`](patterns/pcx.hexpat) | PCX Image format |
| PE | `application/x-dosexec` `application/x-msdownload` | [`patterns/pe.hexpat`](patterns/pe.hexpat) | PE header, COFF header, Standard COFF fields and Windows Specific fields |
| PP | | [`patterns/selinuxpp.hexpat`](patterns/selinuxpp.pat) | SE Linux package |
| PFS0 | | [`patterns/pfs0.hexpat`](patterns/pfs0.hexpat) | Nintendo Switch PFS0 archive (NSP files) |
| PIF | `image/pif` | [`patterns/pif.hexpat`](patterns/pif.hexpat) | PIF Image Format |
| PNG | `image/png` | [`patterns/png.hexpat`](patterns/png.hexpat) | PNG image files |
| PRODINFO | | [`patterns/prodinfo.hexpat`](patterns/prodinfo.hexpat) | Nintendo Switch PRODINFO |
| Protobuf | | [`patterns/protobuf.hexpat`](patterns/protobuf.hexpat) | Google Protobuf encoding |
| psafe3 | | [`patterns/psafe3.hexpat`](patterns/psafe3.hexpat`) | Password Safe V3 |
| PyInstaller | | [`patterns/pyinstaller.hexpat`](patterns/pyinstaller.hexpat) | PyInstaller binray files |
| PYC | | [`patterns/pyc.hexpat`](patterns/pyc.hexpat) | Python bytecode files |
| QBCL | | [`patterns/qbcl.hexpat`](patterns/qbcl.hexpat) | Qubicle voxel scene project file |
| QOI | `image/qoi` | [`patterns/qoi.hexpat`](patterns/qoi.hexpat) | QOI image files |
| RAS | `image/x-sun-raster` | [`patterns/ras.hexpat`](patterns/ras.hexpat) | RAS image files |
| ReFS | | [`patterns/refs.hexpat`](patterns/refs.hexpat) | Microsoft Resilient File System |
| Shell Link | `application/x-ms-shortcut` | [`patterns/lnk.hexpat`](patterns/lnk.hexpat) | Windows Shell Link file format |
| shp | | [`patterns/shp.hexpat`](patterns/shp.hexpat) | ESRI shape file |
| shx | | [`patterns/shx.hexpat`](patterns/shx.hexpat) | ESRI index file |
| SPIRV | | [`patterns/spirv.hexpat`](patterns/spirv.hexpat) | SPIR-V header and instructions |
| STL | `model/stl` | [`patterns/stl.hexpat`](patterns/stl.hexpat) | STL 3D Model format |
| StuffItV5 | `application/x-stuffit` | [`patterns/sit5.hexpat`](patterns/sit5.hexpat) | StuffIt V5 archive |
| SWF | |[`patterns/swf.hexpat`](patterns/swf.hexpat) | Shockwave Flash file format |
| TAR | `application/x-tar` | [`patterns/tar.hexpat`](patterns/tar.hexpat) | Tar file format |
| TIFF | `image/tiff` | [`patterns/tiff.hexpat`](patterns/tiff.hexpat) | Tag Image File Format |
| TGA | `image/tga` | [`patterns/tga.hexpat`](patterns/tga.hexpat) | Truevision TGA/TARGA image |
| TTF | `font/ttf`, `font/otf` | [`patterns/ttf.hexpat`](patterns/ttf.hexpat) | TrueType and OpenType font format |
| Ubiquiti | | [`patterns/ubiquiti.hexpat`](patterns/ubiquiti.hexpat) | Ubiquiti Firmware (update) image |
| UEFI | | [`patterns/uefi.hexpat`](patterns/uefi.hexpat)` | UEFI structs for parsing efivars |
| UF2 | | [`patterns/uf2.hexpat`](patterns/uf2.hexpat) | [USB Flashing Format](https://github.com/microsoft/uf2) |
| VBMeta | | [`patterns/vbmeta.hexpat`](patterns/vbmeta.hexpat) | Android VBMeta image |
| VDF | | [`patterns/vdf.hexpat`](patterns/vdf.hexpat) | Binary Value Data Format (.vdf) files |
| VHDX | | [`patterns/vhdx.hexpat`](patterns/vhdx.hexpat) | Microsoft Hyper-V Virtual Hard Disk format |
| WAV | `audio/x-wav` | [`patterns/wav.hexpat`](patterns/wav.hexpat) | RIFF header, WAVE header, PCM header |
| WAS | | [`patterns\was_oskasoftware.hexpat`](patterns\was_oskasoftware.hexpat) | Oska Software DeskMates WAS/WA3 (WAVE/MP3 Set) file
| WAD | | [`patterns/wad.hexpat`](patterns/wad.hexpat) | DOOM WAD Archive |
| XBEH | `audio/x-xbox-executable` | [`patterns/xbeh.hexpat`](patterns/xbeh.hexpat) | Xbox executable |
| XCI | | [`patterns/xci.hexpat`](patterns/xci.hexpat) | Nintendo Switch XCI cartridge ROM |
| XGT | | [`patterns/xgt.hexpat`](patterns/xgstexture.hexpat) | Exient XGS Engine Texture |
| Xilinx BIT | | [`patterns/xilinx_bit.hexpat`](patterns/xilinx_bit.hexpat) | Xilinx FPGA Bitstreams |
| Xilinx Bootgen | | [`patterns/xilinx_bootgen.hexpat`](patterns/xilinx_bootgen.hexpat) | Xilinx ZynqMP Boot Images |
| ZIP | `application/zip` | [`patterns/zip.hexpat`](patterns/zip.hexpat) | End of Central Directory Header, Central Directory File Headers |
| ZLIB | `application/zlib` | [`patterns/zlib.hexpat`](patterns/zlib.hexpat) | ZLIB compressed data format |
| ZSTD | `application/zstd` | [`patterns/zstd.hexpat`](patterns/zstd.hexpat) | Zstandard compressed data format |
### Scripts
@@ -83,9 +144,9 @@ Hex patterns, include patterns and magic files for the use with the ImHex Hex Ed
| Name | Path | Description |
|------|------|-------------|
| libstd | [`includes/std/*`](includes/std) | Pattern Language Standard Libaray |
| libtype | [`includes/type/*`](includes/type) | Various custom types with special formatters |
| libhex | [`includes/hex/*`](includes/hex) | Functions to interact with ImHex |
| libstd | [`includes/std/*`](includes/std) | Pattern Language Standard Library |
| libtype | [`includes/type/*`](includes/type) | Various custom types with special formatters |
### Yara rules
@@ -120,6 +181,7 @@ Hex patterns, include patterns and magic files for the use with the ImHex Hex Ed
| ASCII+OEM | [`encodings/ascii_oem.tbl`](encodings/ascii_oem.tbl) | ASCII encoding with Windows OEM characters |
| Baltic ISO | [`encodings/baltic_iso.tbl`](encodings/baltic_iso.tbl) | Baltic ISO encoding |
| Baltic Windows | [`encodings/baltic_windows.tbl`](encodings/baltic_windows.tbl) | Baltic Windows encoding |
| Big5 (Traditional Chinese) | [`encodings/big5.tbl`](encodings/big5.tbl) | Big5 encoding for Traditional Chinese |
| Cyrillic ISO | [`encodings/cyrillic_iso.tbl`](encodings/cyrillic_iso.tbl) | Cyrillic ISO encoding |
| Cyrillic Windows | [`encodings/cyrillic_windows.tbl`](encodings/cyrillic_windows.tbl) | Cyrillic Windows encoding |
| Cyrillic KOI8-R | [`encodings/cyrillic_koi8_r.tbl`](encodings/cyrillic_koi8_r.tbl) | Cyrillic KOI8-R encoding (Russian Characters) |
@@ -140,22 +202,29 @@ Hex patterns, include patterns and magic files for the use with the ImHex Hex Ed
| JIS 0213 | [`encodings/jis_x_0213.tbl`](encodings/jis_x_0213.tbl) | JIS X 0213 encoding in UTF-8 |
| Macintosh | [`encodings/macintosh.tbl`](encodings/macintosh.tbl) | Macintosh character encoding used by the Kermit protocol |
| Pokémon (English, Generation 1) | [`encodings/pokegen1_en.tbl`](encodings/pokegen1_en.tbl) | Character encoding used by the English generation 1 Pokémon games |
| Pokémon (English, Generation 3) | [`encodings/pokegen3_en.tbl`](encodings/pokegen3_en.tbl) | Character encoding used by the English generation 3 Pokémon games |
| Shift-JIS UTF-8 | [`encodings/shiftjis.tbl`](encodings/shiftjis.tbl) | Shift-JIS encoding in UTF-8 |
| Thai | [`encodings/thai.tbl`](encodings/thai.tbl) | Thai character encoding |
| Turkish ISO | [`encodings/turkish_iso.tbl`](encodings/turkish_iso.tbl) | Turkish ISO encoding |
| Turkish Windows | [`encodings/turkish_windows.tbl`](encodings/turkish_windows.tbl) | Turkish Windows encoding |
| UTF-8 | [`encodings/utf8.tbl`](encodings/utf8.tbl) | UTF-8 encoding |
| Vietnamese | [`encodings/vietnamese.tbl`](encodings/vietnamese.tbl) | Vietnamese character encoding |
> import custom encoding from File -> Import... -> Custome Encoding File
### Data Processor Nodes
| Name | Path | Description |
|------|------|-------------|
| Caesar Cipher | [`nodes/caesar.hexnode`](nodes/caesar.hexnode) | Simple adjustable per-byte Caecar Cipher (ROT) |
| XOR Cipher | [`nodes/xor.hexnode`](nodes/xor.hexnode) | XORs a input with a repeating XOR pad |
## Contributing
### Themes
If you want to contribute a file to the database, please make a PR which adds it to the right folder and adds a new entry to the table in this readme.
To take advantage of the automatic pattern testing, please consider adding a test file named `<pattern_name>.hexpat.<extension>` to the `/tests/patterns/test_data` directory. Try to keep this file as small as possible so the repository doesn't become excessively large
Thanks a lot :)
| Name | Path | Description |
|------|------|-------------|
| Visual Studio Dark | [`themes/vs_dark.json`](themes/vs_dark.json) | Theme similar to Visual Studio's Dark theme |
| Solarized Dark | [`themes/solarized_dark.json`](themes/solarized_dark.json) | Solarized Dark Theme |
| Catppuccin Latte | [`themes/catppuccin-latte.json`](themes/catppuccin-latte.json) | Catppuccin Latte Flavor (Light Theme) |
| Catppuccin Frappe | [`themes/catppuccin-frappe.json`](themes/catppuccin-frappe.json) | Catppuccin Frappe Flavor (Dark Theme) |
| Catppuccin Macchiato | [`themes/catppuccin-macchiato.json`](themes/catppuccin-macchiato.json) | Catppuccin Macchiato Flavor (Dark Theme) |
| Catppuccin Mocha | [`themes/catppuccin-mocha.json`](themes/catppuccin-mocha.json) | Catppuccin Mocha Flavor (Dark Theme) |

18594
encodings/big5.tbl Normal file

File diff suppressed because it is too large Load Diff

256
encodings/pokegen3_en.tbl Normal file
View File

@@ -0,0 +1,256 @@
00
01=À
02=Á
03=Â
04=Ç
05=È
06=È
07=Ê
08=Ë
09=Ì
0a
0b=Î
0c=Ï
0d=Ò
0e=Ó
0f=Ô
10=Œ
11=Ù
12=Ú
13=Û
14=Ñ
15=ß
16=à
17=á
18
19=ç
1a=è
1b=é
1c=ê
1d=ë
1e=ì
1f
20=î
21=ï
22=ò
23=ó
24=ô
25=œ
26=ù
27=ú
28=û
29=ñ
2a=º
2b=ª
2c=ᵉʳ
2d=&
2e=+
2f
30
31
32
33
34
35==
36
37
38
39
3a
3b
3c
3d
3e
3f
40
41
42
43
44
45
46
47
48
49
4a
4b
4c
4d
4e
4f
50
51=¿
52=¡
53=Pk
54=Mn
55=Po
56=Ké
57
58
59
5a=Í
5b=%
5c=(
5d=)
5e
5f
60
61
62
63
64
65
66
67
68=â
69
6a
6b
6c
6d
6e
6f=í
70
71
72
73
74
75
76
77
78
79=⬆
7a=⬇
7b=⬅
7c=➡
7d
7e
7f
80
81
82
83
84
85
86
87
88
89
8a
8b
8c
8d
8e
8f
90
91
92
93
94
95
96
97
98
99
9a
9b
9c
9d
9e
9f
a0
a1=0
a2=1
a3=2
a4=3
a5=4
a6=5
a7=6
a8=7
a9=8
aa=9
ab=!
ac=?
ad=.
ae=-
af=・
b0
b1=“
b2=”
b3=
b4=
b5=♂
b6=♀
b7
b8=,
b9=×
ba=/
bb=A
bc=B
bd=C
be=D
bf=E
c0=F
c1=G
c2=H
c3=I
c4=J
c5=K
c6=L
c7=M
c8=N
c9=O
ca=P
cb=Q
cc=R
cd=S
ce=T
cf=U
d0=V
d1=W
d2=X
d3=Y
d4=Z
d5=a
d6=b
d7=c
d8=d
d9=e
da=f
db=g
dc=h
dd=i
de=j
df=k
e0=l
e1=m
e2=n
e3=o
e4=p
e5=q
e6=r
e7=s
e8=t
e9=u
ea=v
eb=w
ec=x
ed=y
ee=z
ef=▶
f0=:
f1=Ä
f2=Ö
f3=Ü
f4=ä
f5=ö
f6=ü
f7
f8
f9
fa
fb
fc
fd
fe
ff

File diff suppressed because it is too large Load Diff

View File

@@ -1,29 +1,50 @@
#pragma once
#include <hex/impl/imhex_check.pat>
import hex.impl.imhex_check;
namespace hex::core {
/*!
Core intrinsic functions to interact with the ImHex Hex Editor
*/
namespace auto hex::core {
/**
A type representing a selection in the hex editor
*/
struct Selection {
bool valid;
u64 address, size;
};
/**
Returns the current selection in the hex editor
@return The current selection
*/
fn get_selection() {
u128 result = builtin::hex::core::get_selection();
u128 selection = builtin::hex::core::get_selection();
Selection result;
if (result == u128(-1)) {
if (selection == u128(-1)) {
result.valid = false;
result.address = 0x00;
result.size = 0x00;
} else {
result.valid = true;
result.address = result >> 64;
result.size = result & u64(-1);
result.address = selection >> 64;
result.size = selection & u64(-1);
}
return result;
};
}
/**
Add a file to the Virtual Filesystem
@param path The name of the file
@param pattern The pattern associated with the file
*/
fn add_virtual_file(str path, auto pattern)
{
builtin::hex::core::add_virtual_file(path, pattern);
};
}

View File

@@ -1,11 +1,75 @@
#pragma once
#include <hex/impl/imhex_check.pat>
import hex.impl.imhex_check;
namespace hex::dec {
import std.mem;
/*!
Library to allow decoding of more complex values
*/
namespace auto hex::dec {
/**
Demangles a mangled name into a human readable name
@param mangled_name The mangled name
@return The demangled name
*/
fn demangle(str mangled_name) {
return builtin::hex::dec::demangle(mangled_name);
};
/**
Decompresses the bytes of a pattern into a section using the zlib algorithm
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@param window_size The window size passed to zlib
@return true if successful, false otherwise
*/
fn zlib_decompress(ref auto pattern, std::mem::Section section, u64 window_size = 0) {
return builtin::hex::dec::zlib_decompress(pattern, section, window_size);
};
/**
Decompresses the bytes of a pattern into a section using the bzip algorithm
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@return true if successful, false otherwise
*/
fn bzip_decompress(ref auto pattern, std::mem::Section section) {
return builtin::hex::dec::bzip_decompress(pattern, section);
};
/**
Decompresses the bytes of a pattern into a section using the LZMA algorithm
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@return true if successful, false otherwise
*/
fn lzma_decompress(ref auto pattern, std::mem::Section section) {
return builtin::hex::dec::lzma_decompress(pattern, section);
};
/**
Decompresses the bytes of a pattern into a section using the zstd algorithm
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@return true if successful, false otherwise
*/
fn zstd_decompress(ref auto pattern, std::mem::Section section) {
return builtin::hex::dec::zstd_decompress(pattern, section);
};
/**
Decompresses the bytes of a pattern into a section using the lz4 algorithm
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@param frame Whether the data is framed or not
@return true if successful, false otherwise
*/
fn lz4_decompress(ref auto pattern, std::mem::Section section, bool frame = true) {
return builtin::hex::dec::lz4_decompress(pattern, section, frame);
};
}

View File

@@ -1,9 +1,18 @@
#pragma once
#include <hex/impl/imhex_check.pat>
import hex.impl.imhex_check;
namespace hex::http {
/*!
Library to do HTTP requests
*/
namespace auto hex::http {
/**
Performs a HTTP GET request to the given URL and returns the response body
@param url URL to perform the request to
@return Response body
*/
fn get(str url) {
return builtin::hex::http::get(url);
};

43
includes/hex/provider.pat Normal file
View File

@@ -0,0 +1,43 @@
#pragma once
import hex.impl.imhex_check;
/*!
Library to interact with the currently loaded provider.
*/
namespace auto hex::prv {
/**
Queries information from the currently loaded provider. The kind of information that's available depends on the provider that's loaded
> **Available information**
> - File Provider
> - `file_path() -> str`
> - `file_name() -> str`
> - `file_extension() -> str`
> - `creation_time() -> time_t`
> - `access_time() -> time_t`
> - `modification_time() -> time_t`
> - `permissions() -> u16`
> - Disk Provider
> - `file_path() -> str`
> - `sector_size() -> u128`
> - GDB Provider
> - `ip() -> str`
> - `port() -> u16`
> - Process Memory Provider
> - `region_address(regionName) -> u64`
> - `region_size(regionName) -> u64`
> - `process_id() -> u32`
> - `process_name() -> str`
@param category Information category
@param argument Extra argument to pass along
*/
fn get_information(str category, str argument = "") {
return builtin::hex::prv::get_information(category, argument);
};
}

View File

@@ -1,23 +1,30 @@
#pragma once
#include <std/io.pat>
#include <std/mem.pat>
import std.io;
import std.mem;
#include <hex/impl/imhex_check.pat>
#include <hex/dec.pat>
import hex.impl.imhex_check;
import hex.dec;
namespace hex::type {
/*!
Types to automatically decode mangled names
*/
namespace auto hex::type {
/**
A mangled name string that gets demangled when displayed
*/
struct MangledName {
char value[];
} [[sealed, format("hex::type::impl::format_mangled_name")]];
namespace impl {
fn format_mangled_name(ref MangledName name) {
return hex::dec::demangle(name.value);
};
}
}

View File

@@ -1,9 +1,52 @@
#pragma once
namespace std {
import std.sys;
import std.core;
/*!
The array library contains a helper type to make it easier to create multi-dimensional arrays
and pass arrays to functions as parameters.
*/
namespace auto std {
/**
Simple one dimensional array wrapper
@tparam T The array types
@tparam Size Number of entries in the array
*/
struct Array<T, auto Size> {
T data[Size] [[inline]];
} [[format("std::impl::format_array")]];
/**
Simple array wrapper for an array with a size in bytes
@tparam T The array types
@tparam NumBytes Number of bytes the array contains
*/
struct ByteSizedArray<T, auto NumBytes> {
u64 startAddress = $;
T array[while($ - startAddress < NumBytes)] [[inline]];
std::assert($ - startAddress == NumBytes, "Not enough bytes available to fit a whole number of types");
} [[format("std::impl::format_array")]];
/**
An interface type for getting the index of the currently processed element in an array. This is a nice wrapper around `std::core::array_index()`
To use it, inherit from it and use the `this.index` field to get the index of the current element
*/
struct IIndexed {
const u64 index = std::core::array_index();
};
}
namespace impl {
fn format_array(ref auto array) {
return "[ ... ]";
};
}
}

View File

@@ -1,39 +1,67 @@
#pragma once
#include <std/limits.pat>
import std.limits;
namespace std::bit {
/*!
This library contains various helper functions for common bit operations.
*/
fn popcount(u128 x) {
x = (x & (std::limits::u128_max() / 3)) + ((x >> 1) & (std::limits::u128_max() / 3));
x = (x & (std::limits::u128_max() / 5)) + ((x >> 2) & (std::limits::u128_max() / 5));
x = (x & (std::limits::u128_max() / 17)) + ((x >> 4) & (std::limits::u128_max() / 17));
return x % 0xFF;
};
namespace auto std::bit {
fn has_single_bit(u128 x) {
return x != 0 && (x & (x - 1)) == 0;
};
fn bit_ceil(u128 x) {
if (x == 0) return 0;
u8 i;
while ((1 << i) < x)
i = i + 1;
return 1 << i;
};
fn bit_floor(u128 x) {
if (x == 0) return 0;
u8 i;
while ((x >> i) > 0)
i = i + 1;
return 1 << (i - 1);
};
/**
Calculates the number of 1 bits in a given number
@param x The number
@return The number of bits set to 1 in `x`
*/
fn popcount(u128 x) {
const u128 a = 0x55555555555555555555555555555555;
const u128 b = 0x33333333333333333333333333333333;
const u128 c = 0x0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F;
x = (x & a) + ((x >> 1) & a);
x = (x & b) + ((x >> 2) & b);
x = (x & c) + ((x >> 4) & c);
return x % 0xFF;
};
/**
Checks if only a single bit is set to 1 in a given number
@param x The number
@return True if there's a single bit set to 1 in `x`, false otherwise
*/
fn has_single_bit(u128 x) {
return x != 0 && (x & (x - 1)) == 0;
};
/**
Rounds the given number up to the next bigger power of two
@param x The number
@return Next bigger power of two that can fit `x`
*/
fn bit_ceil(u128 x) {
if (x == 0) return 0;
u8 i;
while ((1 << i) < x)
i = i + 1;
return 1 << i;
};
/**
Rounds the given number down to the next bigger power of two
@param x The number
@return Next smaller power of two
*/
fn bit_floor(u128 x) {
if (x == 0) return 0;
u8 i;
while ((x >> i) > 0)
i = i + 1;
return 1 << (i - 1);
};
}

View File

@@ -1,60 +1,176 @@
#pragma once
#include <std/mem.pat>
import std.mem;
namespace std::core {
/*!
The core library contains intrinsics and "compiler magic" functions that
get extra help from the runtime to fulfill their purpose.
*/
namespace auto std::core {
/**
The layout order of each field after byte-endianness has been handled.
`LeftToRight` and `RightToLeft` are deprecated in favor of the clearer `MostToLeastSignificant` and `LeastToMostSignificant` names.
*/
enum BitfieldOrder : u8 {
/**
@warning deprecated
*/
LeftToRight = 0,
RightToLeft = 1
/**
@warning deprecated
*/
RightToLeft = 1,
MostToLeastSignificant = 0,
LeastToMostSignificant = 1
};
/**
Checks if a pattern has a specific attribute assigned to it
@param pattern The pattern to check
@param attribute The attribute's name to check for
*/
fn has_attribute(ref auto pattern, str attribute) {
return builtin::std::core::has_attribute(pattern, attribute);
};
/**
Returns the nth parameter of the attribute of a pattern if it has one
@param pattern The pattern to check
@param attribute The attribute's name to query
@param [index] The parameter index of the attribute to return. Defaults to 0
*/
fn get_attribute_argument(ref auto pattern, str attribute, u32 index = 0) {
return builtin::std::core::get_attribute_argument(pattern, attribute, index);
};
/**
@warning Removed in 1.27.0
*/
fn get_attribute_value(ref auto pattern, str attribute) {
return builtin::std::core::get_attribute_value(pattern, attribute);
builtin::std::error("`std::core::get_attribute_value(pattern, attribute)` has been removed.\nUse `std::core::get_attribute_argument(pattern, attribute, [index])` instead.");
};
/**
Sets the current default endianess.
Any patterns created following this attribute will be created using the set endianess.
@param endian The new default endianess
*/
fn set_endian(std::mem::Endian endian) {
builtin::std::core::set_endian(u32(endian));
};
/**
Gets the current default endianess.
@return The currently set default endianess
*/
fn get_endian() {
return builtin::std::core::get_endian();
};
/**
@warning Removed in 1.28.0
*/
fn set_bitfield_order(BitfieldOrder order) {
builtin::std::core::set_bitfield_order(u32(order));
builtin::std::error("Runtime default bitfield order is no longer supported.\nConsider using `be` or `le` on your bitfield variables,\nor attach attribute `bitfield_order` to the bitfield.");
};
/**
@warning Removed in 1.28.0
*/
fn get_bitfield_order() {
return builtin::std::core::get_bitfield_order();
builtin::std::error("Runtime default bitfield order is no longer supported.\nConsider using `be` or `le` on your bitfield variables,\nor attach attribute `bitfield_order` to the bitfield.");
};
/**
When used inside of a pattern that's being created using a pattern,
returns the current array index that's being processed.
If used outside of an array, always yields 0.
@return The current array index
*/
fn array_index() {
return builtin::std::core::array_index();
};
/**
Queries the number of members of a struct, union or bitfield or the number of entries in an array
@param pattern The pattern to check
@return The number of members in `pattern`
*/
fn member_count(ref auto pattern) {
return builtin::std::core::member_count(pattern);
};
/**
Checks whether or not a given pattern has a member with a given name
@param pattern The pattern to check
@param name The name of the member to look for
@return True if a member called `name` exists, false otherwise
*/
fn has_member(ref auto pattern, str name) {
return builtin::std::core::has_member(pattern, name);
};
/**
Formats a pattern using it's default formatter or its custom formatter function set through
the `[[format]]` or `[[format_read]]` attribute
@param pattern The pattern to format
@return Formatted string representation of `pattern`
*/
fn formatted_value(ref auto pattern) {
return builtin::std::core::formatted_value(pattern);
};
/**
Checks if the given enum value corresponds has a corresponding constant
@param pattern The enum value to check
@return True if pattern has a valid enum representation, false if not
*/
fn is_valid_enum(ref auto pattern) {
return builtin::std::core::is_valid_enum(pattern);
};
/**
Changes the color of the given pattern to a new color
@param pattern The pattern to modify
@param color The RGBA8 color
*/
fn set_pattern_color(ref auto pattern, u32 color) {
builtin::std::core::set_pattern_color(pattern, color);
};
/**
Changes the display name of a given pattern
@param pattern The pattern to modify
@param name The new display name of the pattern
*/
fn set_display_name(ref auto pattern, str name) {
builtin::std::core::set_display_name(pattern, name);
};
/**
Changes the comment attached to a pattern
@param pattern The pattern to modify
@param name The new comment of the pattern
*/
fn set_pattern_comment(ref auto pattern, str comment) {
builtin::std::core::set_pattern_comment(pattern, name);
};
/**
Executes the function with the given name, passing in all given arguments
@param function_name The namespace-prefixed name of the function
@param args Arguments to pass to the function
*/
fn execute_function(str function_name, auto ... args) {
builtin::std::core::execute_function(function_name, args);
};
}

View File

@@ -1,51 +1,116 @@
#pragma once
namespace std::ctype {
/*!
The ctype library has functions to check if a character is part of a specific category
of ASCII characters.
*/
namespace auto std::ctype {
/**
Checks if the given character `c` is a digit between '0' and '9'
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isdigit(char c) {
return c >= '0' && c <= '9';
};
/**
Checks if the given character `c` is a hexadecimal digit between '0' and '9', `A` and `F` or `a` and `f`
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isxdigit(char c) {
return std::ctype::isdigit(c) || (c >= 'A' && c <= 'F') || (c >= 'a' && c <= 'f');
};
/**
Checks if the given character `c` is a upper case letter between 'A' and 'Z'
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isupper(char c) {
return c >= 'A' && c <= 'Z';
};
/**
Checks if the given character `c` is a lower case letter between 'a' and 'z'
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn islower(char c) {
return c >= 'a' && c <= 'z';
};
/**
Checks if the given character `c` is either a upper or lower case letter between 'A' and 'Z' or 'a' and 'z'
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isalpha(char c) {
return std::ctype::isupper(c) || std::ctype::islower(c);
};
/**
Checks if the given character `c` is a upper or lower case letter or a number
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isalnum(char c) {
return std::ctype::isalpha(c) || std::ctype::isdigit(c);
};
/**
Checks if the given character `c` is a space character
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isspace(char c) {
return (c >= 0x09 && c <= 0x0D) || c == 0x20;
};
/**
Checks if the given character `c` is a invisible character
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isblank(char c) {
return c == 0x09 || c == ' ';
};
/**
Checks if the given character `c` has a printable glyph
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isprint(char c) {
return c >= '0' && c <= '~';
};
/**
Checks if the given character `c` is a control code
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn iscntrl(char c) {
return !std::ctype::isprint(c);
};
/**
Checks if the given character `c` has a visible glyph
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn isgraph(char c) {
return std::ctype::isprint(c) && !std::ctype::isspace(c);
};
/**
Checks if the given character `c` is a punctuation character
@param c The character to check
@return True if `c` is part of this range, false otherwise
*/
fn ispunct(char c) {
return std::ctype::isgraph(c) && !std::ctype::isalnum(c);
};

View File

@@ -1,53 +1,120 @@
#pragma once
namespace std::file {
/*!
The File library allows reading and writing from/to external files using
a C-like File IO API.
using Handle = s32;
enum Mode : u8 {
Read = 1,
Write = 2,
Create = 3
};
**These functions are considered dangerous and require the user to manually permit them**
*/
namespace auto std::file {
/**
A handle representing a file that has been opened
*/
using Handle = s32;
/**
The mode to open a file in.
Read opens the file in read-only mode
Write opens the file in read and write mode
Create creates a new file if it doesn't exist and overwrites an existing file
*/
enum Mode : u8 {
Read = 1,
Write = 2,
Create = 3
};
fn open(str path, Mode mode) {
return builtin::std::file::open(path, u32(mode));
};
/**
Opens a file
@param path The path to the file to open
@param mode File open mode
@return Handle to the newly opened file
*/
fn open(str path, Mode mode) {
return builtin::std::file::open(path, u32(mode));
};
fn close(Handle handle) {
builtin::std::file::close(handle);
};
/**
Closes a file handle that has been opened previously
@param handle The handle to close
*/
fn close(Handle handle) {
builtin::std::file::close(handle);
};
fn read(Handle handle, u64 size) {
return builtin::std::file::read(handle, size);
};
/**
Reads the content of a file into a string
@param handle The file handle to read from
@param size Number of bytes to read
@return String containing the read data
*/
fn read(Handle handle, u64 size) {
return builtin::std::file::read(handle, size);
};
fn write(Handle handle, str data) {
return builtin::std::file::write(handle, data);
};
/**
Writes the content of a string into a file
@param handle The file handle to write to
@param data String or Pattern to write to the file
*/
fn write(Handle handle, auto data) {
builtin::std::file::write(handle, data);
};
fn seek(Handle handle, u64 offset) {
builtin::std::file::seek(handle, offset);
};
/**
Sets the current cursor position in the given file handle
@param handle The file handle to set the cursor position in
@param offset The offset to move the cursor to
*/
fn seek(Handle handle, u64 offset) {
builtin::std::file::seek(handle, offset);
};
fn size(Handle handle) {
return builtin::std::file::size(handle);
};
/**
Queries the size of a file
@param handle The handle of the file to get the size of
@return The file's size
*/
fn size(Handle handle) {
return builtin::std::file::size(handle);
};
fn resize(Handle handle, u64 size) {
builtin::std::file::resize(handle, size);
};
/**
Resizes a file
@param handle The handle of the file to resize
*/
fn resize(Handle handle, u64 size) {
builtin::std::file::resize(handle, size);
};
fn flush(Handle handle) {
builtin::std::file::remove(handle);
};
/**
Flushes changes made to a file to disk
@param handle The handle of the file to flush
*/
fn flush(Handle handle) {
builtin::std::file::flush(handle);
};
fn remove(Handle handle) {
builtin::std::file::remove(handle);
};
/**
Deletes a file from disk. This will also automatically close this file
@param handle The handle of the file to delete
*/
fn remove(Handle handle) {
builtin::std::file::remove(handle);
};
/**
Create all directories for the provided path
@param path The path for which all directories should be created
*/
fn create_directories(str path) {
builtin::std::file::create_directories(path);
};
}

View File

@@ -1,35 +1,89 @@
#pragma once
namespace std::fxpt {
/*!
Library for doing arithmetic with fixed point numbers and converting them from/to floating point numbers.
*/
using fixed = s128;
namespace auto std::fxpt {
fn to_float(fixed fxt, u32 precision) {
return double(fxt) / double((1 << precision));
};
fn to_fixed(double flt, u32 precision) {
return fixed((flt * (1 << precision)));
};
/**
A fixed point value
*/
using fixed = s128;
fn change_precision(fixed value, u32 start_precision, u32 end_precision) {
return std::fxpt::to_fixed(std::fxpt::to_float(value, start_precision), end_precision);
};
fn add(fixed a, fixed b, u32 precision) {
return a + b;
};
fn subtract(fixed a, fixed b, u32 precision) {
return a - b;
};
fn multiply(fixed a, fixed b, u32 precision) {
return (a * b) / (1 << precision);
};
fn divide(fixed a, fixed b, u32 precision) {
return (a << precision) / b;
};
/**
Converts a fixed point value into a floating point value
@param fxt The fixed point value to convert
@param precision The bits of precision the new value should have
@return The floating point representation of fxt
*/
fn to_float(fixed fxt, u32 precision) {
return double(fxt) / double((1 << precision));
};
/**
Converts a floating point value into a fixed point value
@param flt The floating point value to convert
@param precision The bits of precision the new value should have
@return The fixed point representation of flt
*/
fn to_fixed(double flt, u32 precision) {
return s128((flt * (1 << precision)));
};
/**
Changes the number of bits used to represent the decimal part of the given fixed point number
@param value The fixed point value to convert
@param start_precision The current number of bits used
@param end_precision The new number of bits used
@return `value` as a new fixed point number with `end_precision` bits of precision
*/
fn change_precision(fixed value, u32 start_precision, u32 end_precision) {
return std::fxpt::to_fixed(std::fxpt::to_float(value, start_precision), end_precision);
};
/**
Adds two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the addition of `a` and `b`
*/
fn add(fixed a, fixed b, u32 precision) {
return a + b;
};
/**
Subtracts two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the subtraction of `a` and `b`
*/
fn subtract(fixed a, fixed b, u32 precision) {
return a - b;
};
/**
Multiplies two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the multiplication of `a` and `b`
*/
fn multiply(fixed a, fixed b, u32 precision) {
return (a * b) / (1 << precision);
};
/**
Divides two fixed point numbers with a given precision together
@param a First fixed point number
@param b Second fixed point number
@param precision The precision of `a` and `b`
@return Result of the division of `a` and `b`
*/
fn divide(fixed a, fixed b, u32 precision) {
return (a << precision) / b;
};
}

View File

@@ -1,9 +1,65 @@
#pragma once
namespace std::hash {
/*!
The hash library contains various data hash functions
*/
fn crc32(ref auto pattern, u32 init, u32 poly) {
return builtin::std::hash::crc32(pattern, init, poly);
namespace auto std::hash {
/**
Calculates the CRC8 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC8 hash of
@param init The CRC8 init value
@param poly The CRC8 polynomial
@param xorout The CRC8 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC8 hash
*/
fn crc8(ref auto pattern, u8 init, u8 poly, u8 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc8(pattern, init, poly, xorout, reflect_in, reflect_out);
};
/**
Calculates the CRC16 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC16 hash of
@param init The CRC16 init value
@param poly The CRC16 polynomial
@param xorout The CRC16 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC16 hash
*/
fn crc16(ref auto pattern, u16 init, u16 poly, u16 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc16(pattern, init, poly, xorout, reflect_in, reflect_out);
};
/**
Calculates the CRC32 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC32 hash of
@param init The CRC32 init value
@param poly The CRC32 polynomial
@param xorout The CRC32 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC32 hash
*/
fn crc32(ref auto pattern, u32 init, u32 poly, u32 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc32(pattern, init, poly, xorout, reflect_in, reflect_out);
};
/**
Calculates the CRC64 hash of the bytes inside of a given pattern
@param pattern The pattern to calculate the CRC64 hash of
@param init The CRC64 init value
@param poly The CRC64 polynomial
@param xorout The CRC64 XOR-Out value
@param reflect_in Whether or not the input bytes should be reflected
@param reflect_out Whether or not the output should be reflected
@return Calculated CRC64 hash
*/
fn crc64(ref auto pattern, u64 init, u64 poly, u64 xorout, bool reflect_in, bool reflect_out) {
return builtin::std::hash::crc64(pattern, init, poly, xorout, reflect_in, reflect_out);
};
}

View File

@@ -1,20 +1,45 @@
#pragma once
namespace std {
/*!
The IO library allows formatting strings and outputting text to the console
*/
namespace auto std {
/**
Formats the given arguments using the format string and prints the result to the console
This function uses the C++20 `std::format` or libfmt's `fmt::format` syntax.
@param fmt Format string
@param args Values to use in the formatting
*/
fn print(str fmt, auto ... args) {
builtin::std::print(fmt, args);
};
/**
Formats the given arguments using the format string and returns the result as a string
This function uses the C++20 `std::format` or libfmt's `fmt::format` syntax.
@param fmt Format string
@param args Values to use in the formatting
@return The formatted string
*/
fn format(str fmt, auto ... args) {
return builtin::std::format(fmt, args);
};
/**
Aborts evaluation of the code immediately and prints a error message to the console
@param message The message to print
*/
fn error(str message) {
builtin::std::error(message);
};
/**
Prints a warning message to the console
@param message The message to print
*/
fn warning(str message) {
builtin::std::warning(message);
};

View File

@@ -1,85 +1,169 @@
#pragma once
namespace std::limits {
fn u8_min() {
return u8(0);
};
fn u8_max() {
return u8(-1);
};
/*!
Library to calculate the minimum and maximum values that fit into a given data type
*/
namespace auto std::limits {
/**
Returns the minimum value that can be stored in a `u8`.
@return Minimum value
*/
fn u8_min() {
return u8(0);
};
/**
Returns the maximum value that can be stored in a `u8`.
@return Maximum value
*/
fn u8_max() {
return u8(-1);
};
/**
Returns the minimum value that can be stored in a `s8`.
@return Minimum value
*/
fn s8_min() {
return -s8((std::limits::u8_max() / 2)) - 1;
};
fn s8_max() {
return s8((std::limits::u8_max() / 2));
};
return -s8((std::limits::u8_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s8`.
@return Maximum value
*/
fn s8_max() {
return s8((std::limits::u8_max() / 2));
};
fn u16_min() {
return u16(0);
};
fn u16_max() {
return u16(-1);
};
fn s16_min() {
return -s16((std::limits::u16_max() / 2)) - 1;
};
fn s16_max() {
return s16((std::limits::u16_max() / 2));
};
fn u32_min() {
return u32(0);
};
fn u32_max() {
return u32(-1);
};
fn s32_min() {
return -s32((std::limits::u32_max() / 2)) - 1;
};
fn s32_max() {
return s32((std::limits::u32_max() / 2));
};
fn u64_min() {
return u64(0);
};
fn u64_max() {
return u64(-1);
};
fn s64_min() {
return -s64((std::limits::u64_max() / 2)) - 1;
};
fn s64_max() {
return s64((std::limits::u64_max() / 2));
};
fn u128_min() {
return u128(0);
};
fn u128_max() {
return u128(-1);
};
fn s128_min() {
return -s128((std::limits::u128_max() / 2)) - 1;
};
fn s128_max() {
return s128((std::limits::u128_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u16`.
@return Minimum value
*/
fn u16_min() {
return u16(0);
};
/**
Returns the maximum value that can be stored in a `u16`.
@return Maximum value
*/
fn u16_max() {
return u16(-1);
};
/**
Returns the minimum value that can be stored in a `s16`.
@return Minimum value
*/
fn s16_min() {
return -s16((std::limits::u16_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s16`.
@return Maximum value
*/
fn s16_max() {
return s16((std::limits::u16_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u32`.
@return Minimum value
*/
fn u32_min() {
return u32(0);
};
/**
Returns the maximum value that can be stored in a `u32`.
@return Maximum value
*/
fn u32_max() {
return u32(-1);
};
/**
Returns the minimum value that can be stored in a `s32`.
@return Minimum value
*/
fn s32_min() {
return -s32((std::limits::u32_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s32`.
@return Maximum value
*/
fn s32_max() {
return s32((std::limits::u32_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u64`.
@return Minimum value
*/
fn u64_min() {
return u64(0);
};
/**
Returns the maximum value that can be stored in a `u64`.
@return Maximum value
*/
fn u64_max() {
return u64(-1);
};
/**
Returns the minimum value that can be stored in a `s64`.
@return Minimum value
*/
fn s64_min() {
return -s64((std::limits::u64_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s64`.
@return Maximum value
*/
fn s64_max() {
return s64((std::limits::u64_max() / 2));
};
/**
Returns the minimum value that can be stored in a `u128`.
@return Minimum value
*/
fn u128_min() {
return u128(0);
};
/**
Returns the maximum value that can be stored in a `u128`.
@return Maximum value
*/
fn u128_max() {
return u128(-1);
};
/**
Returns the minimum value that can be stored in a `s128`.
@return Minimum value
*/
fn s128_min() {
return -s128((std::limits::u128_max() / 2)) - 1;
};
/**
Returns the maximum value that can be stored in a `s128`.
@return Maximum value
*/
fn s128_max() {
return s128((std::limits::u128_max() / 2));
};
}

View File

@@ -1,109 +1,340 @@
#pragma once
namespace std::math {
import std.mem;
fn min(auto a, auto b) {
if (a < b)
return a;
else
return b;
};
fn max(auto a, auto b) {
if (a > b)
return a;
else
return b;
};
/*!
Library containing more advanced mathematical operations.
*/
fn clamp(auto x, auto min, auto max) {
if (x < min)
return min;
else if (x > max)
return max;
else
return x;
};
namespace auto std::math {
fn abs(auto x) {
if (x < 0)
return -x;
else
return x;
};
/**
Compares the values `a` and `b` with each other and returns the smaller of the two
@param a First value
@param b Second value
@return `a` if `a` is smaller than `b`, otherwise `b`
*/
fn min(auto a, auto b) {
if (a < b)
return a;
else
return b;
};
/**
Compares the values `a` and `b` with each other and returns the bigger of the two
@param a First value
@param b Second value
@return `a` if `a` is bigger than `b`, otherwise `b`
*/
fn max(auto a, auto b) {
if (a > b)
return a;
else
return b;
};
fn sign(auto x) {
if (x > 0)
return 1;
else if (x < 0)
return -1;
else
return 0;
};
/**
Clamps the value of `x` between `min` and `max`.
@param x Value
@param min Minimum value
@param max Maximum value
@return `min` if `x` is smaller than `min`, `max` if `x` is bigger than `max`, `x` otherwise
*/
fn clamp(auto x, auto min, auto max) {
if (x < min)
return min;
else if (x > max)
return max;
else
return x;
};
fn copy_sign(auto x, auto y) {
if (y >= 0)
return std::math::abs(x);
else
return -std::math::abs(x);
};
/**
Returns the absolute value of `x`.
@param x Value
@return `x` if `x` is positive, `-x` otherwise
*/
fn abs(auto x) {
if (x < 0)
return -x;
else
return x;
};
fn factorial(u128 x) {
u128 result;
result = x;
while (x > 1) {
x = x - 1;
result = result * x;
}
return result;
};
/**
Returns the sign of `x`.
@param x Value
@return `1` if `x` is positive, `-1` if `x` is negative, `0` if `x` is zero
*/
fn sign(auto x) {
if (x > 0)
return 1;
else if (x < 0)
return -1;
else
return 0;
};
fn comb(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / (std::math::factorial(k) * std::math::factorial(n - k));
};
/**
Copies the sign of `y` to `x`.
@param x Value
@param y Value
@return `x` if `y` is positive, `-x` if `y` is negative
*/
fn copy_sign(auto x, auto y) {
if (y >= 0)
return std::math::abs(x);
else
return -std::math::abs(x);
};
fn perm(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / std::math::factorial(n - k);
};
/**
Calculates the factorial of `x`.
@param x Value
@return Factorial of `x`
*/
fn factorial(u128 x) {
u128 result;
result = x;
while (x > 1) {
x = x - 1;
result = result * x;
}
return result;
};
/**
Calculates the binomial coefficient of `n` and `k`.
@param n Value
@param k Value
@return Binomial coefficient of `n` and `k`
*/
fn comb(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / (std::math::factorial(k) * std::math::factorial(n - k));
};
/**
Calculates the permutation of `n` and `k`.
@param n Value
@param k Value
@return Permutation of `n` and `k`
*/
fn perm(u128 n, u128 k) {
if (k > n)
return 0;
else
return std::math::factorial(n) / std::math::factorial(n - k);
};
/**
Floors the value of `value`.
@param value Value
@return `value` floored
*/
fn floor(auto value) { return builtin::std::math::floor(value); };
/**
Ceils the value of `value`.
@param value Value
@return `value` ceiled
*/
fn ceil(auto value) { return builtin::std::math::ceil(value); };
/**
Rounds the value of `value`.
@param value Value
@return `value` rounded
*/
fn round(auto value) { return builtin::std::math::round(value); };
/**
Truncates the value of `value`.
@param value Value
@return `value` truncated
*/
fn trunc(auto value) { return builtin::std::math::trunc(value); };
fn floor(auto value) { return builtin::std::math::floor(value); };
fn ceil(auto value) { return builtin::std::math::ceil(value); };
fn round(auto value) { return builtin::std::math::round(value); };
fn trunc(auto value) { return builtin::std::math::trunc(value); };
/**
Calculates the logarithm of `value` with base 10.
@param value Value
@return Logarithm of `value` with base 10
*/
fn log10(auto value) { return builtin::std::math::log10(value); };
fn log10(auto value) { return builtin::std::math::log10(value); };
fn log2(auto value) { return builtin::std::math::log2(value); };
fn ln(auto value) { return builtin::std::math::ln(value); };
/**
Calculates the logarithm of `value` with base 2.
@param value Value
@return Logarithm of `value` with base 2
*/
fn log2(auto value) { return builtin::std::math::log2(value); };
fn fmod(auto value) { return builtin::std::math::fmod(value); };
fn pow(auto base, auto exp) { return builtin::std::math::pow(base, exp); };
fn sqrt(auto value) { return builtin::std::math::sqrt(value); };
fn cbrt(auto value) { return builtin::std::math::cbrt(value); };
/**
Calculates the natural logarithm of `value`.
@param value Value
@return Logarithm of `value` with base `e`
*/
fn ln(auto value) { return builtin::std::math::ln(value); };
fn sin(auto value) { return builtin::std::math::sin(value); };
fn cos(auto value) { return builtin::std::math::cos(value); };
fn tan(auto value) { return builtin::std::math::tan(value); };
/**
Calculates the floating point modulus of `value`.
@param value Value
@return Floating point modulus of `value`
*/
fn fmod(auto value) { return builtin::std::math::fmod(value); };
fn asin(auto value) { return builtin::std::math::asin(value); };
fn acos(auto value) { return builtin::std::math::acos(value); };
fn atan(auto value) { return builtin::std::math::atan(value); };
fn atan2(auto value) { return builtin::std::math::atan2(value); };
/**
Calculates the value of `base` raised to the power of `exp`.
@param base Base
@param exp Exponent
@return `base` raised to the power of `exp`
*/
fn pow(auto base, auto exp) { return builtin::std::math::pow(base, exp); };
fn sinh(auto value) { return builtin::std::math::sinh(value); };
fn cosh(auto value) { return builtin::std::math::cosh(value); };
fn tanh(auto value) { return builtin::std::math::tanh(value); };
/**
Calculates the value of the natural number `e` raised to the power of `value`.
@param value Exponent
@return `e` raised to the power of `value`
*/
fn exp(auto value) { return builtin::std::math::exp(value); };
fn asinh(auto value) { return builtin::std::math::asinh(value); };
fn acosh(auto value) { return builtin::std::math::acosh(value); };
fn atanh(auto value) { return builtin::std::math::atanh(value); };
/**
Calculates the square root of `value`.
@param value Value
@return Square root of `value`
*/
fn sqrt(auto value) { return builtin::std::math::sqrt(value); };
/**
Calculates the cubic root of `value`.
@param value Value
@return Cubic root of `value`
*/
fn cbrt(auto value) { return builtin::std::math::cbrt(value); };
/**
Calculates the sine of `value`.
@param value Angle value in radians
@return Sine of `value`
*/
fn sin(auto value) { return builtin::std::math::sin(value); };
/**
Calculates the cosine of `value`.
@param value Angle value in radians
@return Cosine of `value`
*/
fn cos(auto value) { return builtin::std::math::cos(value); };
/**
Calculates the tangent of `value`.
@param value Angle value in radians
@return Tangent of `value`
*/
fn tan(auto value) { return builtin::std::math::tan(value); };
/**
Calculates the arc sine of `value`.
@param value Angle value in radians
@return Arc sine of `value`
*/
fn asin(auto value) { return builtin::std::math::asin(value); };
/**
Calculates the arc cosine of `value`.
@param value Value
@return Arc cosine of `value` in radians
*/
fn acos(auto value) { return builtin::std::math::acos(value); };
/**
Calculates the arc tangent of `value`.
@param value Value
@return Arc tangent of `value` in radians between `-pi/2` and `pi/2`
*/
fn atan(auto value) { return builtin::std::math::atan(value); };
/**
Calculates the arc tangent of `value`.
@param y Value representing the proportion of the y-coordinate
@param x Value representing the proportion of the x-coordinate.
@return Arc tangent of `value` in radians between `-pi` and `pi`
*/
fn atan2(auto y, auto x) { return builtin::std::math::atan2(y, x); };
/**
Calculates the hyperbolic sine of `value`.
@param value Angle value in radians
@return Hyperbolic sine of `value`
*/
fn sinh(auto value) { return builtin::std::math::sinh(value); };
/**
Calculates the hyperbolic cosine of `value`.
@param value Angle value in radians
@return Hyperbolic cosine of `value`
*/
fn cosh(auto value) { return builtin::std::math::cosh(value); };
/**
Calculates the hyperbolic tangent of `value`.
@param value Angle value in radians
@return Hyperbolic tangent of `value`
*/
fn tanh(auto value) { return builtin::std::math::tanh(value); };
/**
Calculates the arc hyperbolic sine of `value`.
@param value Value
@return Arc hyperbolic sine of `value`
*/
fn asinh(auto value) { return builtin::std::math::asinh(value); };
/**
Calculates the arc hyperbolic cosine of `value`.
@param value Value
@return Arc hyperbolic cosine of `value`
*/
fn acosh(auto value) { return builtin::std::math::acosh(value); };
/**
Calculates the arc hyperbolic tangent of `value`.
@param value Value
@return Arc hyperbolic tangent of `value`
*/
fn atanh(auto value) { return builtin::std::math::atanh(value); };
/**
Options to use with the `std::math::accumulate` function.
*/
enum AccumulateOperation : u8 {
Add = 0,
Multiply = 1,
Modulo = 2,
Min = 3,
Max = 4
};
/**
Calculates the sum of all values in the specified memory range.
@param start Start address
@param end End address
@param valueSize Size of each value in bytes
@param [section] Section to use
@param [operation] Operation to use. Defaults to addition
@param [endian] Endianness to use. Defaults to native
@return Sum of all values in the specified memory range
*/
fn accumulate(u128 start, u128 end, u128 valueSize, std::mem::Section section = 0, AccumulateOperation operation = AccumulateOperation::Add, std::mem::Endian endian = std::mem::Endian::Native) {
return builtin::std::math::accumulate(start, end, valueSize, section, u128(operation), u128(endian));
};
}

View File

@@ -1,19 +1,57 @@
#pragma once
namespace std::mem {
/*!
Library for doing raw memory accesses and other low-level operations.
*/
namespace auto std::mem {
namespace impl {
struct MagicSearchImpl<auto Magic, T> {
s128 address = builtin::std::mem::find_string_in_range(0, $, builtin::std::mem::size(), Magic);
if (address < 0)
break;
$ = address;
try {
T data [[inline]];
} catch {
T data;
}
};
}
/**
A Handle for a custom Section
*/
using Section = u128;
/**
The Endianess of a value
*/
enum Endian : u8 {
Native = 0,
Big = 1,
Little = 2
};
/**
Function that returns true if the cursor position is at the end of the memory
This is usually used in while-sized arrays in the form of `u8 array[while(!std::mem::eof())]`
@return True if the cursor is at the end of the memory
*/
fn eof() {
return $ >= std::mem::size();
return $ >= (std::mem::base_address() + std::mem::size());
};
/**
Aligns the given value to the given alignment
@param alignment The alignment to align to
@param value The value to align
@return The aligned value
*/
fn align_to(u128 alignment, u128 value) {
u128 remainder = value % alignment;
@@ -21,80 +59,223 @@ namespace std::mem {
};
/**
Gets the base address of the data
@return The base address of the memory
*/
fn base_address() {
return builtin::std::mem::base_address();
};
/**
Gets the size of the data
@return The size of the memory
*/
fn size() {
return builtin::std::mem::size();
};
/**
Finds a sequence of bytes in the data
@param occurrence_index The index of the occurrence to find
@param bytes The bytes to find
@return The address of the sequence
*/
fn find_sequence(u128 occurrence_index, auto ... bytes) {
return builtin::std::mem::find_sequence_in_range(occurrence_index, builtin::std::mem::base_address(), builtin::std::mem::size(), bytes);
const u128 address = builtin::std::mem::base_address();
return builtin::std::mem::find_sequence_in_range(occurrence_index, address, address + builtin::std::mem::size(), bytes);
};
/**
Finds a sequence of bytes in a specific region of the data
@param occurrence_index The index of the occurrence to find
@param offsetFrom The offset from which to start searching
@param offsetTo The offset to which to search
@param bytes The bytes to find
@return The address of the sequence
*/
fn find_sequence_in_range(u128 occurrence_index, u128 offsetFrom, u128 offsetTo, auto ... bytes) {
return builtin::std::mem::find_sequence_in_range(occurrence_index, offsetFrom, offsetTo, bytes);
};
/**
Finds a string in the data
@param occurrence_index The index of the occurrence to find
@param string The string to find
@return The address of the sequence
*/
fn find_string(u128 occurrence_index, str string) {
const u128 address = builtin::std::mem::base_address();
return builtin::std::mem::find_string_in_range(occurrence_index, address, address + builtin::std::mem::size(), string);
};
/**
Finds a string in a specific region of the data
@param occurrence_index The index of the occurrence to find
@param offsetFrom The offset from which to start searching
@param offsetTo The offset to which to search
@param string The string to find
@return The address of the sequence
*/
fn find_string_in_range(u128 occurrence_index, u128 offsetFrom, u128 offsetTo, str string) {
return builtin::std::mem::find_string_in_range(occurrence_index, offsetFrom, offsetTo, string);
};
/**
Reads a unsigned value from the memory
@param address The address to read from
@param size The size of the value to read
@param [endian] The endianess of the value to read. Defaults to native
@return The value read
*/
fn read_unsigned(u128 address, u8 size, Endian endian = Endian::Native) {
return builtin::std::mem::read_unsigned(address, size, u32(endian));
};
/**
Reads a signed value from the memory
@param address The address to read from
@param size The size of the value to read
@param [endian] The endianess of the value to read. Defaults to native
@return The value read
*/
fn read_signed(u128 address, u8 size, Endian endian = Endian::Native) {
return builtin::std::mem::read_signed(address, size, u32(endian));
};
fn read_string(u128 address, u8 size) {
/**
Reads a string value from the memory
@param address The address to read from
@param size The size of the value to read
@return The value read
*/
fn read_string(u128 address, u128 size) {
return builtin::std::mem::read_string(address, size);
};
/**
Reads a number of bits from the specified bit offset within the specified byte
@param byteOffset The byte offset within the data
@param bitOffset The bit offset to start the read at within the current byte
@param bitSize The total number of bits to read
@return A u128 containing the value read
*/
fn read_bits(u128 byteOffset, u128 bitOffset, u64 bitSize) {
byteOffset += bitOffset >> 3;
bitOffset = bitOffset & 0x7;
return builtin::std::mem::read_bits(byteOffset, bitOffset, bitSize);
};
/**
Creates a new custom section with the given name
@param name The name of the section
@return The handle to the section
*/
fn create_section(str name) {
return builtin::std::mem::create_section(name);
};
/**
Deletes a custom section
@param section The handle to the section
*/
fn delete_section(Section section) {
builtin::std::mem::delete_section(section);
};
/**
Gets the size of a custom section
@param section The handle to the section
@return The size of the section
*/
fn get_section_size(Section section) {
return builtin::std::mem::get_section_size(section);
};
/**
Changes the size of a custom section
@param section The handle to the section
@param size The new size of the section
*/
fn set_section_size(Section section, u128 size) {
builtin::std::mem::set_section_size(section, size);
};
/**
Copies a range of bytes from one section into another
@param from_section The section to copy from
@param from_address The address to copy from
@param to_section The section to copy to
@param to_address The address to copy to
@param size The size of the range to copy
*/
fn copy_section_to_section(Section from_section, u64 from_address, Section to_section, u64 to_address, u64 size) {
builtin::std::mem::copy_to_section(from_section, from_address, to_section, to_address, size);
};
/**
Copies a range of bytes from the main section into a custom section
@param value The pattern whose bytes should be copied
@param to_section The section to copy to
@param to_address The address to copy to
*/
fn copy_value_to_section(ref auto value, Section to_section, u64 to_address) {
builtin::std::mem::copy_value_to_section(value, to_section, to_address);
};
struct MagicSearch<auto Magic, T> {
if ($ < (std::mem::size() - std::string::length(Magic) - 1)) {
char __potentialMagic__[std::string::length(Magic)] [[hidden, no_unique_address]];
if (__potentialMagic__ == Magic) {
T data [[inline]];
} else {
padding[1];
continue;
}
} else {
padding[1];
continue;
}
/**
Returns the current bit offset when inside of a bitfield.
@return The current bit offset between 0 and 7
*/
fn current_bit_offset() {
return builtin::std::mem::current_bit_offset();
};
/**
Searches for a sequence of bytes and places the given type at that address
@tparam Magic The magic sequence to search for
@tparam T The type to place at the address
*/
struct MagicSearch<auto Magic, T> {
std::mem::impl::MagicSearchImpl<Magic, T> impl[while(!std::mem::eof())] [[inline]];
};
/**
Reinterprets a value as a different one
@tparam From The type to reinterpret from
@tparam To The type to reinterpret to
*/
union Reinterpreter<From, To> {
From from;
To to;
};
/**
Aligns the cursor to the given alignment
@tparam alignment The alignment to align to
*/
struct AlignTo<auto Alignment> {
padding[Alignment - ((($ - 1) % Alignment) + 1)];
} [[hidden, sealed]];
/**
A type representing a sequence of bytes without any specific meaning
@tparam Size The size of the sequence
*/
struct Bytes<auto Size> {
u8 bytes[Size];
} [[sealed, format("std::mem::impl::format_bytes")]];
namespace impl {
fn format_bytes(auto bytes) {
return "";
};
}
}

View File

@@ -1,17 +1,69 @@
#pragma once
namespace std::ptr {
import std.mem;
/*!
The Pointer library contains helper functions to deal with pointer types.
The `relative_to` functions are meant to be used with the `[[pointer_base]]` attribute
*/
namespace auto std::ptr {
/**
Use the offset of the current pointer as start address
@param offset The pointer's value
@return The new pointer base
*/
fn relative_to_pointer(u128 offset) {
return $;
};
/**
Use the offset of the pointer's parent as start address
@param offset The pointer's value
@return The new pointer base
*/
fn relative_to_parent(u128 offset) {
return addressof(parent);
};
/**
Use the end of the file as pointer base address and use its value as offset backwards from there
@param offset The pointer's value
@return The new pointer base
*/
fn relative_to_end(u128 offset) {
return std::mem::size() - offset * 2;
};
}
/**
A nullable pointer, generic over both the pointee type and pointer type.
By nullable, we mean that if the pointer's value is zero (`0x0`), then the
value will appear as padding rather than a pointer to something, but
if the pointer's value is non-zero, that non-zero value will be treated as
a pointer of type `PointerTy` which points to an element of type `PointeeTy`.
Example:
A struct field called `p_myInfo` which is a nullable 64-bit pointer to an
element of type `MyInfoTy` would be written as:
```
struct MyStruct {
std::ptr::NullablePtr<MyInfoTy, u64> p_myInfo;
}
```
*/
struct NullablePtr<PointeeTy, PointerTy> {
// `pointerValue` is `no_unique_address` because we don't want to advance
// the current memory location after reading the value of the pointer itself;
// we want to examine the value at this address to determine what should be
// displayed. It's also `hidden` so the editor only displays either thee
// padding or the populated pointer/pointee field.
PointerTy pointerValue [[no_unique_address, hidden]];
if (pointerValue == 0x0) {
padding[sizeof(PointerTy)];
} else {
PointeeTy *data : PointerTy;
}
};
}

81
includes/std/random.pat Normal file
View File

@@ -0,0 +1,81 @@
#pragma once
import std.limits;
/*!
Library to generate random numbers. Supports various different distribution types.
*/
namespace auto std::random {
/**
Represents the type of distribution to use to generate a random number
*/
enum Distribution : u8 {
Uniform = 0,
Normal = 1,
Exponential = 2,
Gamma = 3,
Weibull = 4,
ExtremeValue = 5,
ChiSquared = 6,
Cauchy = 7,
FisherF = 8,
StudentT = 9,
LogNormal = 10,
Bernoulli = 11,
Binomial = 12,
NegativeBinomial = 13,
Geometric = 14,
Poisson = 15
};
/**
Sets the seed of the random number generator
@param seed Seed to use
*/
fn set_seed(u64 seed) {
builtin::std::random::set_seed(seed);
};
/**
Generates a random number using the given distribution with the given parameters.
The random number generator used internally is C++'s std::mt19937_64 Mersenne Twister implementation.
> **Distributions**
> - `Uniform(min, max) -> i128`
> - `Normal(mean, stddev) -> double`
> - `Exponential(lambda) -> double`
> - `Gamma(alpha, beta) -> double`
> - `Weibull(a, b) -> double`
> - `ExtremeValue(a, b) -> double`
> - `ChiSquared(n) -> double`
> - `Cauchy(a, b) -> double`
> - `FisherF(m, n) -> double`
> - `StudentT(n) -> double`
> - `LogNormal(m, s) -> double`
> - `Bernoulli(p) -> bool`
> - `Binomial(t, p) -> i128`
> - `NegativeBinomial(k, p) -> i128`
> - `Geometric(p) -> i128`
> - `Poisson(mean) -> i128`
@param distribution Distribution to use
@param [param1] This parameter depends on the type of distribution used. Defaults to 0
@param [param2] This parameter depends on the type of distribution used. Defaults to 0
*/
fn generate_using(Distribution distribution, auto param1 = 0, auto param2 = 0) {
return builtin::std::random::generate(u32(distribution), param1, param2);
};
/**
Generates a uniformly distributed random number between `min` and `max`
@param [min] Minimum number. Defaults to 0
@param [max] Maximum number. Defaults to `u64_max`
*/
fn generate(u64 min = std::limits::u64_min(), u64 max = std::limits::u64_max()) {
return std::random::generate_using(Distribution::Uniform, min, max);
};
}

View File

@@ -1,72 +1,167 @@
#pragma once
#include <std/io.pat>
import std.io;
import std.mem;
namespace std::string {
/*!
Libray to interact with strings.
*/
struct SizedStringBase<SizeType, DataType> {
SizeType size;
DataType data[size];
} [[sealed, format("std::string::impl::format_sized_string"), transform("std::string::impl::format_sized_string")]];
namespace auto std::string {
using SizedString<SizeType> = SizedStringBase<SizeType, char>;
using SizedString16<SizeType> = SizedStringBase<SizeType, char16>;
/**
Base type for sized strings. Represents a string with its size preceeding it.
@tparam SizeType The type of the size field.
@tparam DataType The type of the characters.
*/
struct SizedStringBase<SizeType, DataType> {
SizeType size;
DataType data[size];
} [[sealed, format("std::string::impl::format_string"), transform("std::string::impl::format_string")]];
namespace impl {
/**
A ASCII string with a prefixed size.
@tparam SizeType The type of the size field.
*/
using SizedString<SizeType> = SizedStringBase<SizeType, char>;
fn format_sized_string(ref auto string) {
return string.data;
};
/**
A UTF-16 string with a prefixed size.
@tparam SizeType The type of the size field.
*/
using SizedString16<SizeType> = SizedStringBase<SizeType, char16>;
}
/**
Base type for null-terminated strings. Represents a string with its size determined by the first 0x00 byte found.
@tparam DataType The type of the characters.
*/
struct NullStringBase<DataType> {
DataType string[while(std::mem::read_unsigned($, sizeof(DataType)) != 0x00)];
DataType null_terminator;
} [[sealed, format("std::string::impl::format_string"), transform("std::string::impl::format_string")]];
fn length(str string) {
return builtin::std::string::length(string);
};
/**
A null-terminated ASCII string.
*/
using NullString = NullStringBase<char>;
fn at(str string, u32 index) {
return builtin::std::string::at(string, index);
};
/**
A null-terminated UTF-16 string.
*/
using NullString16 = NullStringBase<char16>;
fn substr(str string, u32 pos, u32 count) {
return builtin::std::string::substr(string, pos, count);
};
namespace impl {
fn format_string(ref auto string) {
return string.data;
};
}
/**
Gets the length of a string.
@param string The string to get the length of.
@return The length of the string.
*/
fn length(str string) {
return builtin::std::string::length(string);
};
/**
Gets the character at a given index.
@param string The string to get the character from.
@param index The index of the character to get.
@return The character at the given index.
*/
fn at(str string, u32 index) {
return builtin::std::string::at(string, index);
};
/**
Gets a substring of a string.
@param string The string to get the substring from.
@param pos The position of the first character of the substring.
@param count The number of characters to get.
@return The substring.
*/
fn substr(str string, u32 pos, u32 count) {
return builtin::std::string::substr(string, pos, count);
};
fn parse_int(str string, u8 base) {
return builtin::std::string::parse_int(string, base);
};
/**
Converts a string to an integer.
@param string The string to convert.
@param base The base of the number.
@return The integer.
*/
fn parse_int(str string, u8 base) {
return builtin::std::string::parse_int(string, base);
};
fn parse_float(str string) {
return builtin::std::string::parse_float(string);
};
/**
Converts a string to a float.
@param string The string to convert.
@return The float.
*/
fn parse_float(str string) {
return builtin::std::string::parse_float(string);
};
/**
Converts any type to a string.
@param x The value to convert.
@return The string.
*/
fn to_string(auto x) {
return std::format("{}", x);
};
/**
Checks if a string starts with a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string starts with the substring, false otherwise.
*/
fn starts_with(str string, str part) {
return std::string::substr(string, 0, std::string::length(part)) == part;
};
/**
Checks if a string ends with a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string ends with the substring, false otherwise.
*/
fn ends_with(str string, str part) {
return std::string::substr(string, std::string::length(string) - std::string::length(part), std::string::length(part)) == part;
};
fn contains(str a, str b) {
s32 a_len = std::string::length(a);
s32 b_len = std::string::length(b);
/**
Checks if a string contains a given substring.
@param string The string to check.
@param part The substring to check for.
@return True if the string contains the substring, false otherwise.
*/
fn contains(str string, str part) {
s32 string_len = std::string::length(string);
s32 part_len = std::string::length(part);
for (s32 i = 0, i <= (a_len - b_len), i += 1) {
if (std::string::substr(a, i, b_len) == b)
for (s32 i = 0, i <= (string_len - part_len), i += 1) {
if (std::string::substr(string, i, part_len) == part)
return true;
}
return false;
};
fn reverse(str string) {
/**
Reverses a string.
@param string The string to reverse.
@return The reversed string.
*/
fn reverse(str string) {
str result;
s32 i;
@@ -79,68 +174,82 @@ namespace std::string {
return result;
};
/**
Converts a string to upper case.
@param string The string to convert.
@return The converted string.
*/
fn to_upper(str string) {
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'a' && c <= 'z')
result = result + char(c - 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
fn to_lower(str string) {
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'A' && c <= 'Z')
result = result + char(c + 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'a' && c <= 'z')
result = result + char(c - 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
/**
Converts a string to lower case.
@param string The string to convert.
@return The converted string.
*/
fn to_lower(str string) {
str result;
u32 i;
char c;
while (i < std::string::length(string)) {
c = std::string::at(string, i);
if (c >= 'A' && c <= 'Z')
result = result + char(c + 0x20);
else
result = result + c;
i = i + 1;
}
return result;
};
/**
Replaces all occurrences of a substring with another substring.
@param string The string to replace in.
@param pattern The substring to replace.
@param replace The substring to replace with.
@return The string with the replacements.
*/
fn replace(str string, str pattern, str replace) {
u32 string_len, pattern_len, replace_len;
string_len = std::string::length(string);
pattern_len = std::string::length(pattern);
replace_len = std::string::length(replace);
if (pattern_len > string_len)
return string;
str result;
u32 i;
while (i <= (string_len - pattern_len)) {
if (std::string::substr(string, i, pattern_len) == pattern) {
result = result + replace;
i = i + pattern_len;
} else {
result = result + std::string::at(string, i);
i = i + 1;
}
}
return result;
};
s32 string_len = std::string::length(string);
s32 pattern_len = std::string::length(pattern);
if (pattern_len > string_len || pattern_len * string_len == 0 )
return string;
str result;
s32 string_index;
s32 remaining_len = string_len;
while (pattern_len <= remaining_len) {
if (std::string::substr(string, string_index, pattern_len) == pattern) {
result += replace;
string_index += pattern_len;
} else {
result += std::string::at(string, string_index);
string_index += 1;
}
remaining_len = string_len - string_index;
}
result += std::string::substr(string, string_index, remaining_len );
return result;
};
}

View File

@@ -1,15 +1,29 @@
#pragma once
#include <std/io.pat>
import std.io;
namespace std {
/*!
Basic helper functions
*/
namespace auto std {
/**
Asserts that a given value is true. If it's not, abort evaluation and print the given message to the console
@param condition The condition that is required to be true
@param message The message to print in case the assertion doesn't hold
*/
fn assert(bool condition, str message) {
if (!condition) {
std::error(std::format("assertion failed '{0}'", message));
}
};
/**
Asserts that a given value is true. If it's not, print the given message to the console as a warning
@param condition The condition that is required to be true
@param message The message to print in case the assertion doesn't hold
*/
fn assert_warn(bool condition, str message) {
if (!condition) {
std::warning(std::format("assertion failed '{0}'", message));
@@ -17,12 +31,29 @@ namespace std {
};
/**
Queries the value of a set environment variable given it's name
@param name The name of the env variable
@return The value of that variable
*/
fn env(str name) {
return builtin::std::env(name);
};
/**
Returns the number of parameters in a parameter pack.
@param pack The pack to check
@return Number of parameters in `pack`
*/
fn sizeof_pack(auto ... pack) {
return builtin::std::sizeof_pack(pack);
};
/**
Throws an error notifying the developer that the current codepath is not implemented currently.
*/
fn unimplemented() {
std::error("Unimplemented code path reached!");
};
}

View File

@@ -1,121 +1,200 @@
#pragma once
#include <std/io.pat>
import std.io;
namespace std::time {
/*!
Library to handle time and date related operations.
*/
struct Time {
u8 sec;
u8 min;
u8 hour;
u8 mday;
u8 mon;
u16 year;
u8 wday;
u16 yday;
bool isdst;
} [[sealed]];
namespace auto std::time {
union TimeConverter {
Time time;
u128 value;
};
/**
A structured representation of a time and date.
*/
struct Time {
u8 sec;
u8 min;
u8 hour;
u8 mday;
u8 mon;
u16 year;
u8 wday;
u16 yday;
bool isdst;
} [[sealed]];
using EpochTime = u128;
/**
A helper type to convert between Time and u128.
*/
union TimeConverter {
Time time;
u128 value;
};
enum TimeZone : u8 {
Local,
UTC
};
/**
A type to represent a time in seconds since the epoch.
*/
using EpochTime = u32;
bitfield DOSDate {
day: 5;
month: 4;
year: 7;
} [[sealed]];
/**
A type to represent a time zone.
*/
enum TimeZone : u8 {
Local,
UTC
};
bitfield DOSTime {
seconds: 5;
minutes: 6;
hours: 5;
} [[sealed]];
/**
A type to represent a DOS date.
*/
bitfield DOSDate {
day: 5;
month: 4;
year: 7;
} [[sealed]];
namespace impl {
/**
A type to represent a DOS time.
*/
bitfield DOSTime {
seconds: 5;
minutes: 6;
hours: 5;
} [[sealed]];
union DOSDateConverter {
DOSDate date;
u16 value;
};
namespace impl {
union DOSTimeConverter {
DOSTime time;
u16 value;
};
union DOSDateConverter {
DOSDate date;
u16 value;
};
}
union DOSTimeConverter {
DOSTime time;
u16 value;
};
fn epoch() {
return builtin::std::time::epoch();
};
}
fn to_local(EpochTime epoch_time) {
TimeConverter converter;
converter.value = builtin::std::time::to_local(epoch_time);
/**
Returns the current time in seconds since the epoch.
@return The current time in seconds since the epoch.
*/
fn epoch() {
return builtin::std::time::epoch();
};
return converter.time;
};
/**
Converts a time in seconds since the epoch to a local time.
@param epoch_time The time in seconds since the epoch.
@return The local time.
*/
fn to_local(EpochTime epoch_time) {
le TimeConverter converter;
fn to_utc(EpochTime epoch_time) {
TimeConverter converter;
converter.value = builtin::std::time::to_local(epoch_time);
converter.value = builtin::std::time::to_utc(epoch_time);
return converter.time;
};
return converter.time;
};
/**
Converts a time in seconds since the epoch to a UTC time.
@param epoch_time The time in seconds since the epoch.
@return The UTC time.
*/
fn to_utc(EpochTime epoch_time) {
le TimeConverter converter;
fn now(TimeZone time_zone = TimeZone::Local) {
TimeConverter converter;
converter.value = builtin::std::time::to_utc(epoch_time);
if (time_zone == TimeZone::Local)
converter.value = builtin::std::time::to_local(std::time::epoch());
else if (time_zone == TimeZone::UTC)
converter.value = builtin::std::time::to_utc(std::time::epoch());
else
converter.value = 0x00;
return converter.time;
};
return converter.time;
};
/**
Queries the current time in the specified time zone.
@param [time_zone] The time zone to query. Defaults to local.
@return The current time in the specified time zone.
*/
fn now(TimeZone time_zone = TimeZone::Local) {
le TimeConverter converter;
fn to_dos_date(u16 value) {
impl::DOSDateConverter converter;
if (time_zone == TimeZone::Local)
converter.value = builtin::std::time::to_local(std::time::epoch());
else if (time_zone == TimeZone::UTC)
converter.value = builtin::std::time::to_utc(std::time::epoch());
else
converter.value = 0x00;
converter.value = value;
return converter.time;
};
return converter.date;
};
/**
Converts a value to a DOS date.
@param value The value to convert.
@return The DOS date.
*/
fn to_dos_date(u16 value) {
le impl::DOSDateConverter converter;
fn to_dos_time(u16 value) {
impl::DOSTimeConverter converter;
converter.value = value;
converter.value = value;
return converter.date;
};
return converter.time;
};
/**
Converts a value to a DOS time.
@param value The value to convert.
@return The DOS time.
*/
fn to_dos_time(u16 value) {
le impl::DOSTimeConverter converter;
fn format(Time time, str format_string = "%c") {
TimeConverter converter;
converter.time = time;
converter.value = value;
return builtin::std::time::format(format_string, converter.value);
};
return converter.time;
};
fn format_dos_date(DOSDate date, str format_string = "{}/{}/{}") {
return std::format(format_string, date.day, date.month, date.year + 1980);
};
/**
Converts a FILETIME to unix time.
@param value The value to convert.
@return Timestamp formatted as unix time.
*/
fn filetime_to_unix(u64 value) {
return value / 10000000 - 11644473600;
};
fn format_dos_time(DOSTime time, str format_string = "{:02}:{:02}:{:02}") {
return std::format(format_string, time.hours, time.minutes, time.seconds * 2);
};
/**
Formats a time according to the specified format string.
@param time The time to format.
@param [format_string] The format string to use. Defaults to "%c".
@return The formatted time.
*/
fn format(Time time, str format_string = "%c") {
le TimeConverter converter;
converter.time = time;
return builtin::std::time::format(format_string, converter.value);
};
/**
Formats a DOS date according to the specified format string.
@param date The DOS date to format.
@param [format_string] The format string to use. Defaults to "{}/{}/{}".
@return The formatted DOS date.
*/
fn format_dos_date(DOSDate date, str format_string = "{}/{}/{}") {
return std::format(format_string, date.day, date.month, date.year + 1980);
};
/**
Formats a DOS time according to the specified format string.
@param time The DOS time to format.
@param [format_string] The format string to use. Defaults to "{:02}:{:02}:{:02}".
@return The formatted DOS time.
*/
fn format_dos_time(DOSTime time, str format_string = "{:02}:{:02}:{:02}") {
return std::format(format_string, time.hours, time.minutes, time.seconds * 2);
};
}

View File

@@ -1,16 +1,40 @@
#pragma once
#include <std/io.pat>
#include <std/math.pat>
import std.io;
import std.math;
namespace type {
/*!
Types used to change the base of the displayed integer value.
Used like `type::Hex<u32> hexNumber;`, `type::Oct<u16> octalNumber;`
*/
namespace auto type {
/**
Integer type representing a Hexadecimal value. Displays its value in hexadecimal format.
@tparam T Integer type to use
*/
using Hex<T> = T [[format("type::impl::format_hex")]];
/**
Integer type representing a Octal value. Displays its value in octal format.
@tparam T Integer type to use
*/
using Oct<T> = T [[format("type::impl::format_oct")]];
/**
Integer type representing a Decimal value. Displays its value in decimal format.
@tparam T Integer type to use
*/
using Dec<T> = T [[format("type::impl::format_dec")]];
/**
Integer type representing a Binary value. Displays its value in binary format.
@tparam T Integer type to use
*/
using Bin<T> = T [[format("type::impl::format_bin")]];
namespace impl {
namespace impl {
fn format_number(auto value, str fmt) {
bool negative = value < 0;
@@ -26,6 +50,6 @@ namespace type {
fn format_dec(auto value) { return type::impl::format_number(value, "{}"); };
fn format_bin(auto value) { return type::impl::format_number(value, "0b{:08b}"); };
}
}
}

63
includes/type/base64.pat Normal file
View File

@@ -0,0 +1,63 @@
import std.io;
import std.string;
import std.mem;
/*!
Type representing a Base64 encoded string
*/
namespace auto type {
/**
Type representing a Base64 encoded string
@tparam T String type
*/
struct Base64<T> {
T string;
} [[sealed, format("type::impl::transform_base64")]];
namespace impl {
fn get_decoded_value(char c) {
if (c >= 'A' && c <= 'Z') return c - 'A';
if (c >= 'a' && c <= 'z') return c - 'a' + 26;
if (c >= '0' && c <= '9') return c - '0' + 52;
if (c == '+') return 62;
if (c == '/') return 63;
return -1; // Invalid character
};
fn decode_base64(str input) {
u64 inputLength = std::string::length(input);
str result;
s32 val = 0;
s32 bits = -8;
for (u32 i = 0, i < inputLength, i += 1) {
char c = input[i];
if (c == '=')
break;
s32 index = type::impl::get_decoded_value(c);
if (index == -1)
continue;
val = (val << 6) + index;
bits += 6;
if (bits >= 0) {
result += char((val >> bits) & 0xFF);
bits -= 8;
}
}
return result;
};
fn transform_base64(ref auto base64) {
return type::impl::decode_base64(base64.string);
};
}
}

View File

@@ -1,9 +1,17 @@
#pragma once
#include <std/io.pat>
import std.io;
namespace type {
/*!
Type to decode a BCD (Binary Coded Decimal) number
*/
namespace auto type {
/**
Decodes a BCD value where one byte represents a single digit
@tparam Digits Number of digits
*/
struct BCD<auto Digits> {
u8 bytes[Digits];
} [[sealed, format_read("type::impl::format_bcd")]];

View File

@@ -1,9 +1,17 @@
#pragma once
#include <std/io.pat>
import std.io;
import std.core;
namespace type {
/*!
Types to display single bytes using various different representations
*/
namespace auto type {
/**
Type visualizing the value of each individual bit
*/
bitfield Bits {
bit0 : 1;
bit1 : 1;
@@ -13,13 +21,19 @@ namespace type {
bit5 : 1;
bit6 : 1;
bit7 : 1;
} [[format("type::impl::format_bits"), right_to_left]];
} [[format("type::impl::format_bits"), bitfield_order(std::core::BitfieldOrder::LeastToMostSignificant, 8)]];
/**
Type visualizing the value of the two nibbles
*/
bitfield Nibbles {
low : 4;
high : 4;
} [[format("type::impl::format_nibbles")]];
/**
Type representing a single Byte. Decodes the byte as it's hexadeicmal value, individual bits and nibbles
*/
union Byte {
u8 value;
Bits bits;
@@ -27,7 +41,7 @@ namespace type {
} [[format("type::impl::format_byte"), single_color]];
namespace impl {
namespace impl {
fn format_byte(Byte byte) {
return std::format("0x{0:02X} (0b{1:08b}) LSB:{2}, MSB:{3}",
@@ -53,6 +67,6 @@ namespace type {
return std::format("{{ {0:0X}, {1:0X} }}", nibbles.high, nibbles.low);
};
}
}
}

View File

@@ -1,30 +1,62 @@
#pragma once
#include <std/io.pat>
#include <std/core.pat>
import std.io;
import std.core;
namespace type {
/*!
Types representing RGB or RGBA colors. The decoded color will be displayed in their color field
*/
struct RGB8 {
u8 r, g, b;
} [[sealed, format("type::impl::format_color")]];
namespace auto type {
struct RGBA8 {
u8 r, g, b, a;
} [[sealed, format("type::impl::format_color")]];
/**
Type representing a generic RGBA color with a variable number of bits for each color
@tparam R Number of bits used for the red component
@tparam G Number of bits used for the green component
@tparam B Number of bits used for the blue component
@tparam A Number of bits used for the alpha component
*/
bitfield RGBA<auto R, auto G, auto B, auto A> {
r : R;
g : G;
b : B;
if (A > 0) a : A;
} [[sealed, format("type::impl::format_color"), color(std::format("{0:02X}{1:02X}{2:02X}FF", r, g, b))]];
/**
Type representing a generic RGB color with a variable number of bits for each color
@tparam R Number of bits used for the red component
@tparam G Number of bits used for the green component
@tparam B Number of bits used for the blue component
*/
using RGB<auto R, auto G, auto B> = RGBA<R,G,B,0>;
bitfield RGB565 {
r : 5;
g : 6;
b : 5;
} [[sealed, format("type::impl::format_color")]];
/**
Type representing a RGBA color with 8 bits for the red component, 8 bits for green, 8 bits for blue and 8 bits for alpha
*/
using RGBA8 = RGBA<8,8,8,8>;
bitfield RGBA4 {
r : 4;
g : 4;
b : 4;
a : 4;
} [[sealed, format("type::impl::format_color")]];
/**
Type representing a RGB color with 8 bits for the red component, 8 bits for green and 8 bits for blue
*/
using RGB8 = RGB<8,8,8>;
/**
Type representing a RGB color with 5 bits for the red component, 6 bits for green and 5 bits for blue
*/
using RGB565 = RGB<5,6,5>;
/**
Type representing a RGBA color with 4 bits for the red component, 4 bits for green, 4 bits for blue and 4 bits for alpha
*/
using RGB4444 = RGBA<4,4,4,4>;
/**
Type representing a RGBA color with 5 bits for the red component, 5 bits for green, 5 bits for blue and 1 bits for alpha
*/
using RGBA5551 = RGBA<5,5,5,1>;
namespace impl {
@@ -44,5 +76,5 @@ namespace type {
};
}
}
}

View File

@@ -1,11 +1,18 @@
#pragma once
#include <std/io.pat>
#include <std/math.pat>
#include <std/mem.pat>
import std.io;
import std.math;
import std.mem;
namespace type {
/*!
Type representing a 16 bit half precision floating point number
*/
namespace auto type {
/**
Type representing a 16 bit half precision floating point number
*/
using float16 = u16 [[format("type::impl::format_float16")]];
namespace impl {
@@ -15,7 +22,7 @@ namespace type {
float floatValue;
};
fn format_float16(float16 value) {
fn format_float16(float16 value) {
u32 sign = value >> 15;
u32 exponent = (value >> 10) & 0x1F;
u32 mantissa = value & 0x3FF;

42
includes/type/fmt.pat Normal file
View File

@@ -0,0 +1,42 @@
#pragma once
import std.io;
/*!
Type that allows specifying its format value using a format string.
## Usage
The following code reads a u32 from the data and formats it as an upper case hexadecimal value with
a minimum of 8 digits which is prefixed by 0x.
The format string is the same as passed to `std::format()` and follows the libfmt specification.
```rust
type::Formatted<u32, "0x{:08X}"> hex_formatted_integer @ 0x00;
```
*/
namespace auto type {
/**
Arbitrarily formatted type
@tparam T Type to format
@tparam FormatString libfmt format string to format the value
*/
struct Formatted<T, auto FormatString> {
T value;
} [[sealed, format("type::impl::format_formatted"), transform("type::impl::transform_formatted")]];
namespace impl {
fn format_formatted(ref auto formatted) {
return std::format(std::format("{{0:{}}}", formatted.FormatString), formatted.value);
};
fn transform_formatted(ref auto formatted) {
return formatted.value;
};
}
}

View File

@@ -1,10 +1,17 @@
#pragma once
#include <std/io.pat>
import std.io;
namespace type {
/*!
Types to deal with UUIDs (Universally Unique Identifiers) / GUIDs (Globally Unique Identifiers) as described in RFC 4122
*/
struct GUID {
namespace auto type {
/**
Type representing a GUID value
*/
struct GUID {
u32 time_low;
u16 time_mid;
u16 time_high_and_version;
@@ -13,26 +20,31 @@ namespace type {
u8 node[6];
} [[sealed, format("type::impl::format_guid")]];
namespace impl {
/**
Alias name for GUID
*/
using UUID = GUID;
fn format_guid(GUID guid) {
bool valid = ((le u16(guid.time_high_and_version) >> 12) <= 5) && (((guid.clock_seq_and_reserved >> 4) >= 8) || ((guid.clock_seq_and_reserved >> 4) == 0));
return std::format("{}{{{:08X}-{:04X}-{:04X}-{:02X}{:02X}-{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}}}",
valid ? "" : "Invalid ",
le u32(guid.time_low),
le u16(guid.time_mid),
le u16(guid.time_high_and_version),
guid.clock_seq_and_reserved,
guid.clock_seq_low,
guid.node[0],
guid.node[1],
guid.node[2],
guid.node[3],
guid.node[4],
guid.node[5]);
};
namespace impl {
}
fn format_guid(GUID guid) {
bool valid = ((le u16(guid.time_high_and_version) >> 12) <= 5) && (((guid.clock_seq_and_reserved >> 4) >= 8) || ((guid.clock_seq_and_reserved >> 4) == 0));
return std::format("{}{{{:08X}-{:04X}-{:04X}-{:02X}{:02X}-{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}}}",
valid ? "" : "Invalid ",
le u32(guid.time_low),
le u16(guid.time_mid),
le u16(guid.time_high_and_version),
guid.clock_seq_and_reserved,
guid.clock_seq_low,
guid.node[0],
guid.node[1],
guid.node[2],
guid.node[3],
guid.node[4],
guid.node[5]);
};
}
}

View File

@@ -1,19 +1,30 @@
#pragma once
#include <std/io.pat>
import std.io;
import std.string;
namespace type {
/*!
Types used to decode IP addresses
*/
namespace auto type {
/**
A 4 byte IPv4 Address as described in RFC 791
*/
struct IPv4Address {
u8 bytes[4];
} [[sealed, format("type::impl::format_ipv4_address")]];
/**
A 16 byte IPv6 Address as described in RFC 8200
*/
struct IPv6Address {
u16 words[8];
be u16 words[8];
} [[sealed, format("type::impl::format_ipv6_address")]];
namespace impl {
fn format_ipv4_address(IPv4Address address) {
return std::format("{}.{}.{}.{}",
address.bytes[0],
@@ -24,33 +35,33 @@ namespace type {
fn format_ipv6_address(IPv6Address address) {
str result;
bool hadZeros = false;
s8 startIndex = -1;
for (u8 i = 0, i < 8, i += 1) {
if (address.words[i] == 0x00 && !hadZeros) {
hadZeros = true;
startIndex = i;
while (i < 7) {
if (address.words[i + 1] != 0x00)
break;
i += 1;
}
if (startIndex == 0 || i == 7)
result += ":";
} else {
result += std::format("{:04X}", address.words[i]);
result += std::format("{:X}", address.words[i]);
}
result += ":";
}
return std::string::substr(result, 0, std::string::length(result) - 1);
};
}
}

View File

@@ -1,33 +1,72 @@
#pragma once
#include <std/io.pat>
#include <std/mem.pat>
import std.io;
import std.mem;
namespace type {
struct LEB128 {
u8 array[while($ == addressof(this) || std::mem::read_unsigned($-1, 1) & 0x80 != 0)] [[hidden]];
} [[sealed, format("type::impl::format_leb128"), transform("type::impl::transform_leb128")]];
namespace impl {
fn transform_leb128_array(auto array) {
u128 res = array[0] & 0x7f;
for(u8 i = 1, array[i-1] & 0x80 != 0, i+=1) {
res |= u64(array[i] & 0x7f) << 7 * i;
}
return res;
};
fn format_leb128(auto leb128) {
u128 res = type::impl::transform_leb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_leb128(auto leb128) {
return type::impl::transform_leb128_array(leb128.array);
};
}
/*!
Types used to decode Little Endian Base 128 numbers used to store large numbers as space efficiently as possible
*/
namespace auto type {
/**
Base LEB128 type. Use `uLEB128` and `sLEB128` instead.
*/
struct LEB128Base {
u8 array[while($ == addressof(this) || std::mem::read_unsigned($-1, 1) & 0x80 != 0)] [[hidden]];
} [[sealed]];
/**
A unsigned variant of a LEB128 number
*/
using uLEB128 = LEB128Base [[format("type::impl::format_uleb128"), transform("type::impl::transform_uleb128")]];
/**
A signed variant of a LEB128 number
*/
using sLEB128 = LEB128Base [[format("type::impl::format_sleb128"), transform("type::impl::transform_sleb128")]];
/**
Legacy alias for uLEB128
*/
using LEB128 = uLEB128;
namespace impl {
fn transform_uleb128_array(ref auto array) {
u128 res = array[0] & 0x7f;
for(u8 i = 1, array[i-1] & 0x80 != 0, i+=1) {
res |= u128(array[i] & 0x7f) << 7 * i;
}
return res;
};
fn transform_sleb128_array(ref auto array) {
s128 res = type::impl::transform_uleb128_array(array);
if (res & 0x40 != 0) {
res |= ~0 << (sizeof(array) / sizeof(u8)) * 7;
}
return res;
};
fn format_uleb128(ref auto leb128) {
u128 res = type::impl::transform_uleb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_uleb128(ref auto leb128) {
return type::impl::transform_uleb128_array(leb128.array);
};
fn format_sleb128(ref auto leb128) {
s128 res = type::impl::transform_sleb128_array(leb128.array);
return std::format("{} ({:#x})", res, res);
};
fn transform_sleb128(ref auto leb128) {
return type::impl::transform_sleb128_array(leb128.array);
};
}
}

View File

@@ -1,25 +1,32 @@
#pragma once
#include <std/io.pat>
import std.io;
namespace type {
/*!
Types used to decode MAC Addresses
*/
struct MACAddress {
u8 bytes[6];
} [[sealed, format("type::impl::format_mac_address")]];
namespace auto type {
namespace impl {
/**
A MAC Address as used in the Internet Protocol
*/
struct MACAddress {
u8 bytes[6];
} [[sealed, format("type::impl::format_mac_address")]];
namespace impl {
fn format_mac_address(MACAddress address) {
return std::format("{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}",
address.bytes[0],
address.bytes[1],
address.bytes[2],
address.bytes[3],
address.bytes[4],
address.bytes[5]);
};
return std::format("{:02X}:{:02X}:{:02X}:{:02X}:{:02X}:{:02X}",
address.bytes[0],
address.bytes[1],
address.bytes[2],
address.bytes[3],
address.bytes[4],
address.bytes[5]);
};
}
}
}

View File

@@ -1,19 +1,43 @@
#include <std/string.pat>
#include <std/sys.pat>
import std.string;
import std.sys;
import std.io;
import std.ctype;
namespace type {
/*!
Types used to parse and enforce specific magic numbers
*/
struct Magic<auto ExpectedValue> {
namespace auto type
{
fn fm(ref auto value)
{
str result;
for (u32 i = 0, i < sizeof(value), i += 1)
{
char c = value[i];
if (std::ctype::isprint(c))
result += c;
else
result += std::format("\\x{:02X}", u8(c));
}
return std::format("\"{}\"", result);
};
/**
A Magic number. Throws an error if the magic number does not match the expected value
@tparam ExpectedValue A string representing the expected value
*/
struct Magic<auto ExpectedValue>
{
char value[std::string::length(ExpectedValue)];
std::assert(value == ExpectedValue, std::format("Invalid magic value! Expected \"{}\", got \"{}\".", ExpectedValue, value));
std::assert_warn(value == ExpectedValue, std::format("Invalid magic value! Expected {}, got {} at position 0x{:X}", type::fm(ExpectedValue), type::fm(value), $ - std::string::length(ExpectedValue)));
} [[sealed, format("type::impl::format_magic")]];
namespace impl {
fn format_magic(ref auto magic) {
return magic.value;
namespace impl
{
fn format_magic(ref auto magic)
{
return type::fm(magic.value);
};
}
}
}

View File

@@ -1,32 +1,53 @@
#include <std/mem.pat>
import std.mem;
namespace type {
/*!
Types dealing with various kinds of resource paths
*/
struct UnixPathSegment {
char string[while(std::mem::read_unsigned($, 1) != '/' && std::mem::read_unsigned($, 1) != 0x00)];
char separator [[hidden]];
if (separator == 0x00) {
$ -= 1;
break;
}
} [[sealed, format("type::impl::format_unix_path_segment")]];
struct UnixPath {
UnixPathSegment segments[while(true)];
} [[format("type::impl::format_unix_path")]];
namespace impl {
fn format_unix_path_segment(UnixPathSegment segment) {
return segment.string;
};
fn format_unix_path(UnixPath path) {
return std::mem::read_string($, sizeof(path));
};
}
namespace auto type {
}
/**
Type representing a single path segment. Use the `Path` type instead of using this on its own
@tparam Delimeter The delimeter sequence used to separate two path segments
*/
struct PathSegment<auto Delimeter> {
char string[while(std::mem::read_string($, std::string::length(Delimeter)) != Delimeter && std::mem::read_unsigned($, 1) != 0x00)];
char separator [[hidden]];
if (separator == 0x00) {
$ -= 1;
break;
}
} [[sealed, format("type::impl::format_path_segment")]];
/**
A generic type representing a path with an arbitrary delimeter
@tparam Delimeter The delimeter sequence used to separate two path segments
*/
struct Path<auto Delimeter> {
PathSegment<Delimeter> segments[while(true)];
} [[format("type::impl::format_path")]];
/**
A type representing a Unix path using a '/' forwardslash as delimeter
*/
using UnixPath = Path<"/">;
/**
A type representing a DOS path using a '\\' backslash as delimeter
*/
using DOSPath = Path<"\\">;
namespace impl {
fn format_path_segment(ref auto segment) {
return segment.string;
};
fn format_path(ref auto path) {
return std::mem::read_string($, sizeof(path));
};
}
}

View File

@@ -1,13 +1,36 @@
#include <std/io.pat>
import std.io;
namespace type {
/*!
Types used to pretty print size values
*/
namespace auto type {
/**
A generic size type which displays its value in Bytes (or kiB, MiB, GiB, TiB, PiB, EiB if larger)
@tparam T Underlying type
*/
using Size<T> = T [[format("type::impl::size_formatter")]];
/**
A 8 bit size type
*/
using Size8 = Size<u8>;
/**
A 16 bit size type
*/
using Size16 = Size<u16>;
/**
A 32 bit size type
*/
using Size32 = Size<u32>;
/**
A 64 bit size type
*/
using Size64 = Size<u64>;
/**
A 128 bit size type
*/
using Size128 = Size<u128>;
namespace impl {

View File

@@ -1,14 +1,43 @@
#pragma once
#include <std/io.pat>
#include <std/time.pat>
import std.io;
import std.time;
namespace type {
/*!
Types used to decode various different time formats
*/
namespace auto type {
/**
A 32 bit Unix time value
*/
using time32_t = u32 [[format("type::impl::format_time_t")]];
/**
Alias name for `time32_t`
*/
using time_t = time32_t;
/**
A 64 bit Unix time value
*/
using time64_t = u64 [[format("type::impl::format_time_t")]];
using dosdate16_t = u16 [[format("type::impl::format_dosdate16_t")]];
using dostime16_t = u16 [[format("type::impl::format_dostime16_t")]];
/**
A DOS Date value
*/
using DOSDate = u16 [[format("type::impl::format_dosdate")]];
/**
A DOS Time value
*/
using DOSTime = u16 [[format("type::impl::format_dostime")]];
/**
A 64bit FILETIME value
*/
using FILETIME = u64 [[format("type::impl::format_filetime_as_unix")]];
namespace impl {
@@ -16,14 +45,18 @@ namespace type {
return std::time::format(std::time::to_utc(value));
};
fn format_dosdate16_t(u16 value) {
fn format_dosdate(u16 value) {
return std::time::format_dos_date(std::time::to_dos_date(value));
};
fn format_dostime16_t(u16 value) {
fn format_dostime(u16 value) {
return std::time::format_dos_time(std::time::to_dos_time(value));
};
fn format_filetime_as_unix(u64 value) {
return std::time::filetime_to_unix(value);
};
}
}

View File

@@ -1,7 +1,11 @@
#pragma once
/*!
Alias types to make it easier to move template definitions over from 010 Editor to ImHex
*/
// Explicitly don't add these types to the `type` namespace for usability
// namespace type {
// namespace auto type {
// using char = s8;
using byte = s8;

View File

@@ -1,7 +1,11 @@
#pragma once
/*!
Alias definitions for all C stdint and regular data types
*/
// Explicitly don't add these types to the `type` namespace for usability
// namespace type {
// namespace auto type {
using uint8_t = u8;
using uint16_t = u16;

View File

@@ -1,7 +1,11 @@
#pragma once
/*!
Various data types used in the Linux Kernel
*/
// Explicitly don't add these types to the `type` namespace for usability
// namespace type {
// namespace auto type {
using le16 = le u16;
using be16 = be u16;

View File

@@ -1,7 +1,11 @@
#pragma once
/*!
Alias definitions for Rust's data types
*/
// Explicitly don't add these types to the `type` namespace for usability
// namespace type {
// namespace auto type {
// using u8 = u8;
// using u16 = u16;

View File

@@ -1,7 +1,11 @@
#pragma once
/*!
Alias definitions for various type names used in Windows applications
*/
// Explicitly don't add these types to the `type` namespace for usability
// namespace type {
// namespace auto type {
using BYTE = u8;
using WORD = u16;

View File

@@ -1,4 +1,4 @@
# A libmagic database containing definition for PE files used by MS-DOS based systems
# A libmagic database containing definition for PE files used by MS-DOS/Windows based systems
# MS-DOS Portable Executable
0x0 string/b MZ MS-DOS Binary

1141
patterns/3ds.hexpat Normal file

File diff suppressed because it is too large Load Diff

102
patterns/7z.hexpat Normal file
View File

@@ -0,0 +1,102 @@
#pragma description 7z File Format
#pragma MIME application/x-7z-compressed
import std.io;
import std.mem;
import std.math;
enum Type:u8{
startPosition = 0x00, // Start position
sizeStartHeader = 0x20, // Size of start Header
};
enum TypeB:u48{
sevenZipSignature = 0x1C27AFBC7A37, // Defining 7z signature
};
struct StartHeader {
// File signature
u48 signature [[color("FF0000")] ];
// Version format
u16 formatVersion [[color("00FF00")]];
// CRC start header
u32 crcOfTheFollowing20Bytes [[color("0000FF")]];
// Relative offset of End Header
u64 relativeOffsetEndHeader [[color("FFFF00")]];
// Length of End Header
u64 theLengthOfEndHeader [[color("00FFFF")]];
// CRC of End Ender
u32 crcOftheEndHeader [[color("FF00FF")]];
// File size calculation
u64 fileSize = relativeOffsetEndHeader + theLengthOfEndHeader + Type::sizeStartHeader;
// Absolute offset calculation End Header
u64 startEndHeader = relativeOffsetEndHeader + Type::sizeStartHeader;
};
StartHeader startheader @ Type::startPosition;
struct CompressedData {
// Start of compressed data
u8 startOfCompressedData[4] [[color("C0C0C0")]];
};
CompressedData compresseddata @ Type::sizeStartHeader;
struct EndHeader {
// Set padding to place End Header in right position
padding[startheader.relativeOffsetEndHeader];
// Mark link to meta block
u8 linkToMetaBlock [[color("FF0000")]];
// Mark all End Header
char fullEndHeader[startheader.theLengthOfEndHeader] [[color("63954A")]];
// Detect LZMA signature
u64 lzmaSignaturePosition = std::mem::find_sequence_in_range(0, addressof(fullEndHeader), addressof(fullEndHeader) + sizeof(fullEndHeader), 0x23, 0x03, 0x01, 0x01, 0x05, 0x5D);
// Mark positions if LZMA signature was detected
if(lzmaSignaturePosition != 0xFFFFFFFFFFFFFFFF){
u48 lzmaSignature @ lzmaSignaturePosition [[color("0000FF")]];
}
};
EndHeader endheader @ Type::sizeStartHeader;
// Check 7z type
if(startheader.signature == TypeB::sevenZipSignature){
std::print("It is a 7z File Type");
}else{
std::print("The file is not 7z type");
}
std::print("Format Version {} ",startheader.formatVersion);
// Version verification
if(startheader.formatVersion == 0x0400){
std::print("Major Version 0x00 || 0 - Minor Version 0x04 || 4");
}
// Verification of the compressed method is LZMA, Bzip2 or LZMA2
if(compresseddata.startOfCompressedData[0] == Type::startPosition){
std::print("Compressed Method LZMA");
}else if(compresseddata.startOfCompressedData[0] == 0x42){
std::print("Compressed Method Bzip2");
}else{
std::print("Compressed Method LZMA2");
}
std::print("CRC Start Header 0x{:X}",startheader.crcOfTheFollowing20Bytes);
std::print("CRC End Header 0x{:X} ", startheader.crcOftheEndHeader);
std::print("CompressedData length 0x{:X} || {} bytes ",startheader.relativeOffsetEndHeader, startheader.relativeOffsetEndHeader);
std::print("Relative Offset of End Header 0x{:X} || {} bytes",startheader.relativeOffsetEndHeader, startheader.relativeOffsetEndHeader);
std::print("Offset to start End Header 0x{:X} || {} bytes",startheader.startEndHeader,startheader.startEndHeader);
std::print("End Header length 0x{:X} || {} bytes",startheader.theLengthOfEndHeader,startheader.theLengthOfEndHeader);
std::print("File size 0x{:X} || {} bytes",startheader.fileSize, startheader.fileSize);

370
patterns/Crashlvl.hexpat Normal file
View File

@@ -0,0 +1,370 @@
#pragma author AdventureT
#pragma description Crash Bandicoot - Back in Time (fan game) User created level format
#pragma history
#pragma 0.3 2024-05-15 Added support for version 0.95
#pragma 0.2 2023-10-29 Added support for version 0.94c
#pragma 0.1 2023-04-25 Initial support
// Supports all versions till 0.95, newer versions might be compatible!
import type.magic;
import std.string;
import std.array;
struct Header {
type::Magic<"CRASHLVL"> magic;
u8 version;
if (version >= 4) {
std::string::SizedString<u8> gameVersion;
}
std::string::SizedString<u8> levelName;
std::string::SizedString<u8> author;
};
// Background Music
enum BGM : u32 {
None,
BonusBGM,
CrashCreatorBGM,
MainMenuBGM,
WarpRoomBGM,
Level01BGM,
Level02BGM,
Level03BGM,
Level04BGM,
Level05BGM,
SewerBGM,
EgyptBGM,
NBrioBGM
};
enum BGMV2 : u32 {
None,
N_TropyBGM,
CrashCreatorBGM,
MainMenuBGM,
WarpRoomBGM,
Jungle01BGM,
SnowBGM,
RiverBGM,
FutureBGM,
LabBGM,
SewerBGM,
EgyptBGM,
NBrioBGM,
AdventureBGM,
SpyBGM,
ChaseBGM,
TrialsBGM,
SpaceBGM,
Jungle02BGM,
RipperBGM,
TheGreatWallBGM,
RoadToSomewhereBGM,
LavaKoalaBGM,
CortexBGM,
CyberCortexBGM,
ArabicBGM,
N_Tropy2BGM,
JazzBGM,
Space2BGM,
TawnaBonusBGM,
CortexPowerBGM,
ArabicBonusBGM,
EgyptBonusBGM,
FutureBonusBGM,
LostCityBGM,
PolarBGM,
RiverBonusBGM,
RuinsBonusBGM,
SewerBonusBGM,
SnowBonusBGM,
RoadToRuinBGM,
NGinBGM,
Arabia01BGM,
Arabia02BGM,
BashBGM,
Cortex02BGM
};
// v0.95
enum BGMV3 : BGMV2 {
None,
N_TropyBGM,
CrashCreatorBGM,
MainMenuBGM,
WarpRoomBGM,
Jungle01BGM,
SnowBGM,
RiverBGM,
FutureBGM,
LabBGM,
SewerBGM,
EgyptBGM,
NBrioBGM,
AdventureBGM,
SpyBGM,
ChaseBGM,
TrialsBGM,
SpaceBGM,
Jungle02BGM,
RipperBGM,
TheGreatWallBGM,
RoadToSomewhereBGM,
LavaKoalaBGM,
CortexBGM,
CyberCortexBGM,
ArabicBGM,
N_Tropy2BGM,
JazzBGM,
Space2BGM,
TawnaBonusBGM,
CortexPowerBGM,
ArabicBonusBGM,
EgyptBonusBGM,
FutureBonusBGM,
LostCityBGM,
PolarBGM,
RiverBonusBGM,
RuinsBonusBGM,
SewerBonusBGM,
SnowBonusBGM,
RoadToRuinBGM,
NGinBGM,
Arabia01BGM,
Arabia02BGM,
BashBGM,
Cortex02BGM,
MedievalBGM,
PreHistoricBGM,
UnderWaterBGM,
BrioRevisitedBGM,
EgyptChaseBGM,
RuinsLoopBGM,
DingoSynthBGM
};
enum Type : u32 {
Unset,
Flashback,
Trial
};
enum TypeV2 : u32 {
Practice,
Flashback,
Trial,
Adventure
};
enum Skybox : u32 {
Night,
Day,
Storm,
Dawn
};
enum SkyboxV2 : u32 {
Default = 1,
Briolab,
Fort,
Moon,
Toxic,
AboutRight,
Crash1Island,
Arabia,
RoadToRuin,
Black
};
// 0.95
enum SkyboxV3 : u32 {
Default = 1,
Briolab,
Fort,
Moon,
Toxic,
AboutRight,
Crash1Island,
Arabia,
RoadToRuin,
MotorcycleDay,
MotorcycleNoon,
MotorcycleMoon,
MotorcycleNight,
Black
};
enum Scenery : u32 {
None,
FutureTense,
Forest,
Waterfall,
Snow,
Fortress
};
enum SceneryV2 : u32 {
None,
FutureTense,
Forest,
Waterfall,
None2,
Fortress,
None3,
Lava,
TheGreatGate,
Mountain,
KoalaKong,
SunsetVista,
HangemHigh,
Sphynxinator,
Tunnel,
Pipes
};
enum SceneryV4 : u32 {
None,
FutureTense,
Forest,
Waterfall,
Snow,
Fortress,
None2,
Lava,
TheGreatGate,
Mountain,
KoalaKong,
SunsetVista,
HangemHigh,
Sphynxinator,
Tunnel,
Pipes
};
// 0.95
enum SceneryV5 : u32 {
None,
FutureTense,
Forest,
Waterfall,
Snow,
Fortress,
None2,
Lava,
TheGreatGate,
Mountain,
KoalaKong,
SunsetVista,
HangemHigh,
Sphynxinator,
Tunnel,
Pipes,
Medieval,
FutureCity,
TinyArena,
HeavyMachinery,
CrystalCave,
MedievalWithHouses,
CortexBonusChamber
};
enum Weather : u32 {
Default,
Snow,
Rain
};
// 0.95
enum WeatherV2 : u32 {
Default,
Snow,
Rain,
Night,
UnderWater
};
struct Options {
// Type
if (header.version > 1) {
TypeV2 type;
}
else {
Type type;
}
// Skybox
if (header.version > 1) {
if (header.gameVersion == "0.95") {
SkyboxV3 skybox;
}
else {
SkyboxV2 skybox;
}
}
else {
Skybox skybox;
}
// Scenery
if (header.version == 1) {
Scenery scenery;
}
else if (header.version > 1 && header.version < 4) {
SceneryV2 scenery;
}
else {
if (header.gameVersion == "0.95") {
SceneryV5 skybox;
}
else {
SceneryV4 skybox;
}
}
// Weather
if (header.version > 2) {
if (header.gameVersion == "0.95") {
WeatherV2 weather;
}
else {
Weather weather;
}
}
// Background music
if (header.version > 1) {
BGMV2 bgm;
}
else {
BGM bgm;
}
// Time Trial
if (type == Type::Trial) {
u32 timeTrialTicksBronze;
u32 timeTrialTicksSilver;
u32 timeTrialTicksGold;
}
};
struct Object {
std::string::SizedString<u8> objName;
if (header.version > 1) {
u16 x;
u16 y;
bool hasMetafields;
if (hasMetafields) {
u16 numOfMetafields;
u8 metafields[numOfMetafields];
}
}
else {
u32 x;
u32 y;
}
};
struct Objects{
u32 objCount;
std::Array<Object, objCount> objArray;
};
Header header @ $;
Options options @ $;
Objects objects @ $;

198
patterns/adts.hexpat Normal file
View File

@@ -0,0 +1,198 @@
#pragma authors: zhoubo
#pragma version: 0.4
#pragma description: Parse AAC's ADTS(Audio Data Transport Stream) audio files.
#pragma category: Audio
#pragma filemask: *.aac
#pragma IDBytes: FF F //ADTS Syncword
#pragma history:
#pragma 0.4 2024-02-12 zhoubo: Porting from 010 Editor Templates.
#pragma 0.3 2024-02-09 zhoubo: use BitfieldDisablePadding(Unpadded Bitfields) for odd header bytes(7,9 bytes) color, and remove FSeek.
#pragma 0.2 2024-02-05 zhoubo: fix some comment & color.
#pragma 0.1 2022-06-13 zhoubo: Init release. only ADTS, not support ADIF,LATM.
// More information available at:
// 1. https://wiki.multimedia.cx/index.php?title=ADTS
// 2. https://en.wikipedia.org/wiki/Advanced_Audio_Coding
// 3. https://en.wikipedia.org/wiki/AAC
// 4. https://juejin.cn/post/7032170229732442148
#pragma endian big
#include <std/sys.pat>
#include <std/core.pat>
fn GetMPEGVersionComment(auto MPEG_Version)
{
str comment = "";
match (MPEG_Version)
{
(0): comment = "MPEG-4";
(1): comment = "MPEG-2";
}
return comment;
};
fn GetProtectionAbsentComment(auto Protection_absence)
{
str comment = "";
match (Protection_absence)
{
(0): comment = "ADTS has 9 bytes with CRC";
(1): comment = "ADTS has 7 bytes without CRC";
}
return comment;
};
fn GetProfileComment(auto Profile)
{
str comment = "";
match (Profile)
{
(0x00): comment = "AAC Main";
(0x01): comment = "AAC LC (Low Complexity)";
(0x10): comment = "AAC SSR (Scalable Sample Rate)";
(0x11): comment = "AAC LTP (Long Term Prediction)";
}
return comment;
};
fn GetSamplingFrequencyIndexComment(auto Sampling_frequency_index)
{
str comment = "";
match (Sampling_frequency_index)
{
(0x0): comment = "96000Hz";
(0x1): comment = "88200Hz";
(0x2): comment = "64000Hz";
(0x3): comment = "48000Hz";
(0x4): comment = "44100Hz";
(0x5): comment = "32000Hz";
(0x6): comment = "24000Hz";
(0x7): comment = "22050Hz";
(0x8): comment = "16000Hz";
(0x9): comment = "12000Hz";
(0xa): comment = "11025Hz";
(0xb): comment = "8000Hz";
(0xc): comment = "7350Hz";
(0xd): comment = "Reserved";
(0xe): comment = "Reserved";
(0xf): comment = "FORBIDDEN (malformed)";
}
return comment;
};
fn GetChannelConfigurationComment(auto Channel_configuration)
{
str comment = "";
match (Channel_configuration)
{
(0): comment = "Defined in AOT Specifc Config";
(1): comment = "1 channel: front-center";
(2): comment = "2 channels: front-left, front-right";
(3): comment = "3 channels: front-center, front-left, front-right";
(4): comment = "4 channels: front-center, front-left, front-right, back-center";
(5): comment = "5 channels: front-center, front-left, front-right, back-left, back-right";
(6): comment = "6 channels: front-center, front-left, front-right, back-left, back-right, LFE-channel";
(7): comment = "8 channels: front-center, front-left, front-right, side-left, side-right, back-left, back-right, LFE-channel";
}
return comment;
};
fn GetBufferFullnessComment(auto ADTS_buffer_fullness)
{
str comment = "";
match (ADTS_buffer_fullness)
{
(0x7FF): comment = "VBR (most software ignore this field)";
(_): comment = "rate..?? (most software ignore this field)";
}
return comment;
};
fn GetRawDataBlockComment(auto Number_of_AAC_frames_minus_1)
{
str comment = "";
match (Number_of_AAC_frames_minus_1)
{
(0x0): comment = "has 1 AAC data block";
(_): comment = "data block number + 1";
}
return comment;
};
//-----------------------------------
// Define structures used in AAC files
//
// [1.adts_fixed_header information]
// Syncword: 12bits, sync header, always 0xFFF。
// MPEG_Version: 1bit, 0 means MPGE-4, 1 means MPGE-2
// Layer: 2bits, alwayas ”00”
// Protection_absence: 1bit, 0 means ADTS Header 9 bytes; 1 means ADTS Header 7 bytes
// Profile: 2bit, AAC level : Main,LC,SSR,LTP
// Sampling_frequency_index: 4bits, Sampling Frequencies
// Private_bit: 1bit
// Channel_configuration: 3bits, channel number...
// Originality: 1bit
// Home: 1bit
//
// [2.adts_variable_header information]
// Copyright_id_bit: 1bit
// Copyright_id_start: 1bit
// AAC_frame_length: 13bits, AAC frame length : ADTS Header(7 or 9 bytes) + sizeof(AAC Frame)
// ADTS_buffer_fullness: 11bits, 0x7FF means VBR
// Number_of_AAC_frames_minus_1: 2bits, ADTS Frame numbers : Number_of_AAC_frames_minus_1 + 1
//
// [3.CRC information]
// CRC16: 16bits, when Protection_absence=0
bitfield ADTS_HEADER
{
// ADTS_FIXED_HEADER
Syncword : 12 [[color("00FF00"), comment("always 0xFFF")]];
MPEG_Version : 1 [[color("00FF00"), comment(GetMPEGVersionComment(this))]];
Layer : 2 [[color("00FF00"), comment("always 0")]];
Protection_absence : 1 [[color("00FF00"), comment(GetProtectionAbsentComment(this))]];
Profile : 2 [[color("0000FF"), comment(GetProfileComment(this))]];
Sampling_frequency_index : 4 [[color("0000FF"), comment(GetSamplingFrequencyIndexComment(this))]];
Private_bit : 1 [[color("0000FF")]];
Channel_configuration : 3 [[color("0000FF"), comment(GetChannelConfigurationComment(this))]];
Originality : 1 [[color("0000FF")]];
Home : 1 [[color("0000FF")]];
// ADTS_VARIABLE_HEADER
Copyright_id_bit : 1 [[color("0000FF")]];
Copyright_id_start : 1 [[color("0000FF")]];
AAC_frame_length : 13 [[color("0000FF")]];
ADTS_buffer_fullness : 11 [[color("0000FF"), comment(GetBufferFullnessComment(this))]];
Number_of_AAC_frames_minus_1 : 2 [[color("0000FF"), comment(GetRawDataBlockComment(this))]];
// ADTS_CRC_HEADER
if (0 == Protection_absence) // Header with CRC
{
u16 CRC16 [[color("FFFF00")]];
}
else // Header without CRC
{
}
};
struct ADTS_FRAME
{
ADTS_HEADER Header;
if (0 == Header.Protection_absence) // Header with CRC 2 bytes
{
u8 Data[Header.AAC_frame_length - 9] [[color("000000")]];
}
else // Header without CRC
{
u8 Data[Header.AAC_frame_length - 7] [[color("000000")]];
}
};
//---------------------------------------------
ADTS_FRAME adtsFrame[while(!std::mem::eof())] @ 0x00;

View File

@@ -1,7 +1,10 @@
#pragma author WerWolv
#pragma description Nintendo Switch Atmosphère CFW Fatal Error log
#pragma endian little
#include <std/io.pat>
#include <std/sys.pat>
import std.io;
import std.sys;
#define ATMOSPHERE_REBOOT_TO_FATAL_MAGIC "AFE2"
#define ATMOSPHERE_REBOOT_TO_FATAL_MAGIC_1 "AFE1"

66
patterns/ani.hexpat Normal file
View File

@@ -0,0 +1,66 @@
#pragma description Windows animated cursor
#pragma MIME application/x-navi-animation
#pragma endian little
import std.io;
import std.mem;
bitfield HeaderFlags {
icon : 1;
sequence : 1;
padding: 30;
};
struct anih {
u32 struct_size;
u32 stored_frames;
u32 animation_steps;
u32 w;
u32 h;
u32 bits;
u32 planes;
u32 default_jiffies;
HeaderFlags flags;
};
struct rate {
u32 jiffies[parent.size / 4];
};
struct seq {
u32 index[parent.size / 4];
};
struct RiffChunk {
char signature[4];
u32 size;
if (signature == "RIFF" || signature == "LIST") {
char type[4];
RiffChunk chunks[while($ - addressof(type) < size)];
} else if (signature[0] == 'I' && parent.type == "INFO") {
char info[size];
} else if (signature == "anih") {
anih anih [[inline]];
} else if (signature == "rate") {
rate rate [[inline]];
} else if (signature == "seq ") {
seq seq [[inline]];
} else {
std::mem::Bytes<size> data;
}
padding[size % 2];
} [[format_read("read_chunk")]];
fn read_chunk(RiffChunk chunk) {
if (chunk.signature == "RIFF" || chunk.signature == "LIST") {
return std::format("{}<{}> ({})", chunk.signature, chunk.type, chunk.size);
} else {
str ret = std::format("{} ({})", chunk.signature, chunk.size);
try {
ret += std::format(" \"{}\"", chunk.info);
} catch {}
return ret;
}
};
RiffChunk riff @ 0;

View File

@@ -1,8 +1,11 @@
#pragma author WerWolv
#pragma description Static library archive files
#pragma MIME application/x-archive
#include <std/string.pat>
#include <std/mem.pat>
#include <std/sys.pat>
import std.string;
import std.mem;
import std.sys;
struct ARFile {
char file_name[16];

36
patterns/aria2.hexpat Normal file
View File

@@ -0,0 +1,36 @@
#pragma author itsmeow
#pragma description aria2 Control File
#pragma endian big
/*
Format sourced from:
https://aria2.github.io/manual/en/html/technical-notes.html#control-file-aria2-format
Version 0 files are not supported by default
However parsing one is as simple as changing to #pragma endian ittle
and resolving any errors resulting from that by adding be prefixes to struct fields.
*/
struct AriaInFlight {
u32 index;
u32 length;
u32 bitfield_piece_length;
u8 bitfield_p[bitfield_piece_length];
};
struct AriaHeader {
u16 version;
u32 extension;
u32 infohash_length;
u8 infohash[infohash_length];
u32 piece_length;
u64 total_length;
u64 upload_length;
u32 bitfield_length;
u8 bitfield_d[bitfield_length];
u32 inflight_count;
AriaInFlight inflights[inflight_count];
};
AriaHeader aria_header @ 0x00;

View File

@@ -1,7 +1,10 @@
#pragma author WerWolv
#pragma description ARM Cortex M Vector Table Layout
#pragma endian little
#include <std/io.pat>
#include <std/mem.pat>
import std.io;
import std.mem;
#define VTOR 0x00000000
#define EXTERNAL_INTERRUPT_COUNT 64

1
patterns/bastion Submodule

Submodule patterns/bastion added at e6deed433c

View File

@@ -1,73 +1,76 @@
#pragma MIME application/x-bittorrent
#include <std/ctype.pat>
#include <std/mem.pat>
#include <std/string.pat>
namespace bencode {
struct ASCIIDecimal {
char value[while(std::ctype::isdigit(std::mem::read_unsigned($, 1)))];
} [[sealed, format("bencode::format_ascii_decimal"), transform("bencode::format_ascii_decimal")]];
fn format_ascii_decimal(ASCIIDecimal adsasd) {
return std::string::parse_int(adsasd.value, 10);
};
enum Type : u8 {
Integer = 'i',
Dictionary = 'd',
List = 'l',
String0 = '0',
String1 = '1',
String2 = '2',
String3 = '3',
String4 = '4',
String5 = '5',
String6 = '6',
String7 = '7',
String8 = '8',
String9 = '9'
};
struct String {
ASCIIDecimal length;
char separator [[hidden]];
char value[length];
} [[sealed, format("bencode::format_string"), transform("bencode::format_string")]];
fn format_string(String string) {
return string.value;
};
using Bencode;
using Value;
struct DictionaryEntry {
String key;
Value value;
};
struct Value {
Type type;
if (type == Type::Dictionary) {
DictionaryEntry entry[while(std::mem::read_unsigned($, 1) != 'e')];
} else if (type == Type::Integer) {
ASCIIDecimal value;
char end;
} else {
$ -= 1;
String value;
}
};
struct Bencode {
Value value[while(!std::mem::eof())] [[inline]];
char end;
};
}
bencode::Bencode bencode @ 0x00;
#pragma author WerWolv
#pragma description Bencode encoding, used by Torrent files
#pragma MIME application/x-bittorrent
import std.ctype;
import std.mem;
import std.string;
namespace bencode {
struct ASCIIDecimal {
char value[while(std::ctype::isdigit(std::mem::read_unsigned($, 1)))];
} [[sealed, format("bencode::format_ascii_decimal"), transform("bencode::format_ascii_decimal")]];
fn format_ascii_decimal(ASCIIDecimal adsasd) {
return std::string::parse_int(adsasd.value, 10);
};
enum Type : u8 {
Integer = 'i',
Dictionary = 'd',
List = 'l',
String0 = '0',
String1 = '1',
String2 = '2',
String3 = '3',
String4 = '4',
String5 = '5',
String6 = '6',
String7 = '7',
String8 = '8',
String9 = '9'
};
struct String {
ASCIIDecimal length;
char separator [[hidden]];
char value[length];
} [[sealed, format("bencode::format_string"), transform("bencode::format_string")]];
fn format_string(String string) {
return string.value;
};
using Bencode;
using Value;
struct DictionaryEntry {
String key;
Value value;
};
struct Value {
Type type;
if (type == Type::Dictionary) {
DictionaryEntry entry[while(std::mem::read_unsigned($, 1) != 'e')];
} else if (type == Type::Integer) {
ASCIIDecimal value;
char end;
} else {
$ -= 1;
String value;
}
};
struct Bencode {
Value value[while(!std::mem::eof())] [[inline]];
char end;
};
}
bencode::Bencode bencode @ 0x00;

80
patterns/bgcode.hexpat Normal file
View File

@@ -0,0 +1,80 @@
#pragma author Shadlock0133 (aka Aurora)
#pragma description Binary G-Code from Prusa
#include <type/magic.pat>
#include <std/mem.pat>
enum ChecksumType : u16 {
None,
CRC32,
};
enum BlockType : u16 {
FileMetadata,
GCode,
SlicerMetadata,
PrinterMetadata,
PrintMetadata,
Thumbnail,
};
enum Compression : u16 {
None,
Deflate,
Heatshrink11_4,
Heatshrink12_4,
};
enum Encoding : u16 {
Ini,
};
enum ImageFormat : u16 {
Png,
Jpg,
Qoi,
};
struct Header {
type::Magic<"GCDE"> magic;
u32 version;
ChecksumType checksum_type;
};
Header header @ 0;
std::assert(header.version == 1, "only version 1 supported");
struct Block {
BlockType type;
Compression compression;
u32 uncompressed_size;
auto size = uncompressed_size;
if (compression != Compression::None) {
u32 compressed_size;
size = compressed_size;
}
match (type) {
(BlockType::FileMetadata
| BlockType::PrinterMetadata
| BlockType::PrintMetadata
| BlockType::SlicerMetadata): {
Encoding encoding;
}
(BlockType::Thumbnail): {
ImageFormat image_format;
u16 width;
u16 height;
}
(BlockType::GCode): {
u16;
}
(_): { std::assert(false, "unknown type"); }
}
u8 data[size];
match (header.checksum_type) {
(ChecksumType::None): {}
(ChecksumType::CRC32): { u32 checksum; }
}
};
Block blocks[while(!std::mem::eof())] @ $;

View File

@@ -1,5 +1,8 @@
#pragma description OS2/Windows Bitmap files
#pragma MIME image/bmp
#pragma endian little
import std.mem;
struct BitmapFileHeader {
u8 bfType[2];
@@ -9,13 +12,38 @@ struct BitmapFileHeader {
u32 bfOffBits;
};
struct BitmapInfoHeader {
enum Compression : u32 {
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
};
struct CIEXYZ {
u32 ciexyzX;
u32 ciexyzY;
u32 ciexyzZ;
};
struct CIEXYZTRIPLE {
CIEXYZ ciexyzRed;
CIEXYZ ciexyzGreen;
CIEXYZ ciexyzBlue;
};
struct BitmapInfoHeaderV1 {
u32 biSize;
s32 biWidth;
s32 biHeight;
u16 biPlanes;
u16 biBitCount;
u32 biCompression;
Compression biCompression;
u32 biSizeImage;
s32 biXPelsPerMeter;
s32 biYPelsPerMeter;
@@ -23,6 +51,31 @@ struct BitmapInfoHeader {
u32 biClrImportant;
};
struct BitmapInfoHeaderV2 : BitmapInfoHeaderV1 {
u32 biRedMask;
u32 biGreenMask;
u32 biBlueMask;
};
struct BitmapInfoHeaderV3 : BitmapInfoHeaderV2 {
u32 biAlphaMask;
};
struct BitmapInfoHeaderV4 : BitmapInfoHeaderV3 {
u32 biCSType;
CIEXYZTRIPLE biEndpoints;
u32 biGammaRed;
u32 biGammaGreen;
u32 biGammaBlue;
};
struct BitmapInfoHeaderV5 : BitmapInfoHeaderV4 {
u32 biIntent;
u32 biProfileData;
u32 biProfileSize;
u32 biReserved;
};
struct Colors {
u8 blue;
u8 green;
@@ -31,10 +84,21 @@ struct Colors {
};
struct Bitmap {
u8 data[std::mem::size()] [[no_unique_address, hidden]];
BitmapFileHeader bmfh;
BitmapInfoHeader bmih;
// Deduce the header version from its size
u32 bmInfoHeaderSize [[hidden, no_unique_address]];
match (bmInfoHeaderSize) {
(40): BitmapInfoHeaderV1 bmih;
(52): BitmapInfoHeaderV2 bmih;
(56): BitmapInfoHeaderV3 bmih;
(108): BitmapInfoHeaderV4 bmih;
(124): BitmapInfoHeaderV5 bmih;
(_): BitmapInfoHeaderV1 bmih;
}
padding[bmih.biSize - sizeof(bmih)];
if ((bmih.biBitCount != 24) && (bmih.biBitCount != 32))
if (bmih.biBitCount <= 8)
{
if (bmih.biClrUsed > 0 )
Colors rgbq[bmih.biClrUsed];
@@ -46,6 +110,6 @@ struct Bitmap {
u8 lineData[bmih.biSizeImage];
else
u8 lineData[bmfh.bfSize - $];
};
} [[hex::visualize("image", this.data)]];
Bitmap bitmap @ 0x00;

273
patterns/bplist.hexpat Normal file
View File

@@ -0,0 +1,273 @@
#pragma description TODO
import std.math;
import std.core;
import type.magic;
import type.time;
using CFBinaryPlistObject;
enum Marker : u8 {
Null = 0x00,
False = 0x08,
True = 0x09,
Fill = 0x0F,
Int = 0x10,
Real = 0x20,
Date = 0x30,
Data = 0x40,
ASCIIString = 0x50,
Unicode16String = 0x60,
UNK_0x70 = 0x70,
UID = 0x80,
UNK_0x90 = 0x90,
Array = 0xA0,
UNK_0xB0 = 0xB0,
Set = 0xC0,
Dict = 0xD0,
UNK_0xE0 = 0xE0,
UNK_0xF0 = 0xF0
};
fn get_marker_name(u8 marker) {
if (marker == Marker::Null){// null 0000 0000
return "Null ";
}else if (marker == Marker::False){ //bool 0000 1000 // false
return "False";
}else if (marker == Marker::True){//bool 0000 1001 // true
return "True";
}else if (marker == Marker::Fill){ //fill 0000 1111 // fill byte
return "Fill";
}else if (marker & 0xF0 == Marker::Int){ //int 0001 nnnn ... // # of bytes is 2^nnnn, big-endian bytes
return "Int";
}else if (marker & 0xF0 == Marker:: Real){ //real 0010 nnnn ... // # of bytes is 2^nnnn, big-endian bytes
return "Real";
}else if (marker == Marker::Date){ //date 0011 0011 ... // 8 byte float follows, big-endian bytes
return "Date";
}else if (marker & 0xF0 == Marker::Data){ //data 0100 nnnn [int] ... // nnnn is number of bytes unless 1111 then int count follows, followed by bytes
return "Data";
}else if (marker & 0xF0 == Marker::ASCIIString){ //string 0101 nnnn [int] ... // ASCII string, nnnn is # of chars, else 1111 then int count, then bytes
return "ASCIIString";
}else if (marker & 0xF0 == Marker::Unicode16String){ //string 0110 nnnn [int] ... // Unicode string, nnnn is # of chars, else 1111 then int count, then big-endian 2-byte
return "Unicode16String";
}else if (marker & 0xF0 == Marker::UNK_0x70){ //0111 xxxx // unused
return "UNK_0x70";
}else if (marker & 0xF0 == Marker::UID){ //uid 1000 nnnn ... // nnnn+1 is # of bytes
return "UID";
}else if (marker & 0xF0 == Marker::UNK_0x90){ // 1001 xxxx // unused
return "UNK_0x90";
}else if (marker & 0xF0 == Marker::Array){ //array 1010 nnnn [int] objref* // nnnn is count, unless '1111', then int count follows
return "Array";
}else if (marker & 0xF0 == Marker::UNK_0xB0){ //1011 xxxx // unused
return "UNK_0xB0";
}else if (marker & 0xF0 == Marker::Set){ //set 1100 nnnn [int] objref* // nnnn is count, unless '1111', then int count follows
return "Set";
}else if (marker & 0xF0 == Marker::Dict){ //dict 1101 nnnn [int] keyref* objref* // nnnn is count, unless '1111', then int count follows
return "Dict";
}else if (marker & 0xF0 == Marker::UNK_0xE0){ // 1110 xxxx // unused
return "UNK_0xE0";
}else if (marker & 0xF0 == Marker::UNK_0xF0){ //1111 xxxx // unused
return "UNK_0xF0";
}
};
fn format_tag(u8 marker) {
return std::format("{}", get_marker(marker));
};
fn coredata_to_date (double val){
return type::impl::format_time_t(978307200 + val);
};
struct DictElement {
CFBinaryPlistObject key @ offsetTable[parent.objReference.key_refs[std::core::array_index()]].offset;
CFBinaryPlistObject value @ offsetTable[parent.objReference.value_refs[std::core::array_index()]].offset;
};
struct ArrayElement {
CFBinaryPlistObject value @ offsetTable[parent.objReference.value_refs[std::core::array_index()]].offset;
};
struct ObjectReference{
match(trailer.objectRefSize){
(1): {
be u8 key_refs[parent.ObjectLen.size];
be u8 value_refs[parent.ObjectLen.size];
}
(2): {
be u16 key_refs[parent.ObjectLen.size];
be u16 value_refs[parent.ObjectLen.size];
}
(4): {
be u32 key_refs[parent.ObjectLen.size];
be u32 value_refs[parent.ObjectLen.size];
}
(8): {
be u64 key_refs[parent.ObjectLen.size];
be u64 value_refs[parent.ObjectLen.size];
}
}
};
struct ObjectReferenceArray{
match(trailer.objectRefSize){
(1): be u8 value_refs[parent.ObjectLen.size];
(2): be u16 value_refs[parent.ObjectLen.size];
(4): be u32 value_refs[parent.ObjectLen.size];
(8): be u64 value_refs[parent.ObjectLen.size];
}
};
struct ObjectLen{
if (parent.marker_lsb != 0x0F){
u8 size = parent.marker_lsb [[export]];
}else{
CFBinaryPlistObject obj;
if (obj.marker & 0xF0 != Marker::Int){
std::error(std::format("Expects a 'Int' marker. Got 0x{:x} marker, with value {}", obj.marker, obj.value));
}
u128 size = obj.value [[export]];
}
};
struct CFBinaryPlistOffset{
match (trailer.offsetIntSize){
(1): be u8 offset;
(2): be u16 offset;
(4): be u32 offset;
(8): be u64 offset;
}
};
struct CFBinaryPlistObject{
u8 marker [[format("get_marker_name")]];
u8 marker_msb = marker & 0xF0;
u8 marker_lsb = marker & 0x0F;
match (marker_msb){
(0x0): {
match (marker_lsb){
(Marker::Null): {
u8 value = 0x00 [[export]];
}
(Marker::False): {
bool value = false [[export]];
}
(Marker::True): {
bool value = true [[export]];
}
(Marker::Fill): {
//I think the correct implementation is to do nothing here. The marker will be used as padding (Fill) ???
}
(_): {
std::error("Detected unknown marker {}.", marker_msb);
}
}
}
(Marker::Int): {
be u8 size = std::math::pow(2, marker_lsb);
// in format version '00', 1, 2, and 4-byte integers have to be interpreted as unsigned,
// whereas 8-byte integers are signed (and 16-byte when available)
// negative 1, 2, 4-byte integers are always emitted as 8 bytes in format '00'
// integers are not required to be in the most compact possible representation, but only the last 64 bits are significant currently
// Source: https://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c
match (size) {
(1): be u8 value;
(2): be u16 value;
(4): be u32 value;
(8): be s64 value;
(16): be s128 value;
(_): std::error(std::format("Invalid size detected for 'Int' marker. Got size: {}.", size));
}
}
(Marker::Real): {
be u8 size = std::math::pow(2, marker_lsb);
match (size){
(4): be float value;
(8): be double value;
(_): std::error(std::format("Invalid size detected for 'Real' marker. Got size: {}.", size));
}
}
(Marker::Date): {
be double value [[format("coredata_to_date")]];
}
(Marker::Data): {
ObjectLen ObjectLen;
u8 value[ObjectLen.size];
}
(Marker::ASCIIString): {
ObjectLen ObjectLen;
char value[ObjectLen.size];
}
(Marker::Unicode16String): {
ObjectLen ObjectLen;
be char16 value[ObjectLen.size];
}
(Marker::UID): {
//Not 100% sure if this is correct for UID. Need more testing
u8 size = marker_lsb+1;
match (size) {
(1): be u8 value;
(2): be u16 value;
(4): be u32 value;
(8): be u64 value;
(16): be u128 value;
(_): std::error(std::format("Invalid size detected for 'UID' marker. Got size: {}.", size));
}
}
(Marker::Set | Marker::Array): {
ObjectLen ObjectLen;
ObjectReferenceArray objReference;
ArrayElement value[ObjectLen.size];
}
(Marker::Dict): {
ObjectLen ObjectLen;
ObjectReference objReference;
DictElement value[ObjectLen.size];
}
(Marker::UNK_0x70 | Marker::UNK_0x90 | Marker::UNK_0xB0 | Marker::UNK_0xE0 | Marker::UNK_0xF0): {
std::error(std::format("Got unused marker 0x{:x}", marker));
}
(_): {
std::error(std::format("Got unknown marker 0x{:x}", marker));
}
}
};
struct CFBinaryPlistHeader{
type::Magic<"bplist"> magic;
u16 version;
if (version != 0x3030){
std::error("Unsupported version detected. Only version 00 is supported (bplist00).");
}
};
struct CFBinaryPlistTrailer {
u8 unused[5];
u8 sortVersion;
be u8 offsetIntSize;
match (offsetIntSize){
(1|2|4|8): {}
(_): {std::error("Invalid offsetIntSize.");}
}
be u8 objectRefSize;
match (objectRefSize){
(1|2|4|8): {}
(_): {std::error("Invalid objectRefSize.");}
}
be u64 numObjects;
be u64 topObject;
be u64 offsetTableOffset;
};
CFBinaryPlistHeader header @ 0x00;
CFBinaryPlistTrailer trailer @ std::mem::size()-32;
CFBinaryPlistOffset offsetTable[trailer.numObjects] @ trailer.offsetTableOffset;
CFBinaryPlistObject objectTable @ offsetTable[trailer.topObject].offset;

View File

@@ -1,128 +1,132 @@
#pragma MIME application/bson
#include <type/time.pat>
enum Type : u8 {
Double = 0x01,
String = 0x02,
EmbeddedDocument = 0x03,
Array = 0x04,
Binary = 0x05,
Undefined = 0x06,
ObjectId = 0x07,
Boolean = 0x08,
UTCDatetime = 0x09,
Null = 0x0A,
Regex = 0x0B,
DBPointer = 0x0C,
JavaScript = 0x0D,
Symbol = 0x0E,
JavaScriptWithScope = 0x0F,
Int32 = 0x10,
Timestamp = 0x11,
Int64 = 0x12,
Decimal128 = 0x13,
MinKey = 0xFF,
MaxKey = 0x7F
};
enum Subtype : u8 {
GenericBinarySubtype = 0x00,
Function = 0x01,
BinaryOld = 0x02,
UUIDOld = 0x03,
UUID = 0x04,
MD5 = 0x05,
EncryptedBSONValue = 0x06,
CompressedBSONColumn = 0x07,
UserDefined = 0x80
};
struct Binary {
s32 length;
Subtype subtype;
u8 data[length];
};
struct String {
u32 length [[hidden]];
char value[length];
} [[sealed, format("format_string")]];
struct CString {
char value[];
} [[sealed, format("format_string")]];
fn format_string(auto string) {
return string.value;
};
struct ObjectId {
type::time32_t timestamp;
u8 randomValue[5];
u24 counter;
};
struct DBPointer {
String name;
ObjectId value;
};
using Document;
struct Element {
Type type;
CString name;
if (type == Type::Double) {
double value;
} else if (type == Type::String) {
String value;
} else if (type == Type::EmbeddedDocument) {
Document value;
} else if (type == Type::Array) {
Document value;
} else if (type == Type::Binary) {
Binary value;
} else if (type == Type::Undefined) {
/* undefined */
} else if (type == Type::ObjectId) {
ObjectId value;
} else if (type == Type::Boolean) {
bool value;
} else if (type == Type::UTCDatetime) {
type::time64_t value;
} else if (type == Type::Null) {
/* null */
} else if (type == Type::Regex) {
CString regexPattern;
CString regexOptions;
} else if (type == Type::DBPointer) {
DBPointer value;
} else if (type == Type::JavaScript) {
String value;
} else if (type == Type::Symbol) {
String value;
} else if (type == Type::JavaScriptWithScope) {
String value;
} else if (type == Type::Int32) {
s32 value;
} else if (type == Type::Timestamp) {
u64 value;
} else if (type == Type::Int64) {
s64 value;
} else if (type == Type::Decimal128) {
u128 value;
}
};
struct Document {
s32 listLength;
Element elements[while($ < ((addressof(this) + listLength) - 1))];
padding[1];
};
Document document @ 0x00;
#pragma author WerWolv
#pragma description BSON (Binary JSON) format
#pragma MIME application/bson
import std.mem;
import type.time;
enum Type : u8 {
Double = 0x01,
String = 0x02,
EmbeddedDocument = 0x03,
Array = 0x04,
Binary = 0x05,
Undefined = 0x06,
ObjectId = 0x07,
Boolean = 0x08,
UTCDatetime = 0x09,
Null = 0x0A,
Regex = 0x0B,
DBPointer = 0x0C,
JavaScript = 0x0D,
Symbol = 0x0E,
JavaScriptWithScope = 0x0F,
Int32 = 0x10,
Timestamp = 0x11,
Int64 = 0x12,
Decimal128 = 0x13,
MinKey = 0xFF,
MaxKey = 0x7F
};
enum Subtype : u8 {
GenericBinarySubtype = 0x00,
Function = 0x01,
BinaryOld = 0x02,
UUIDOld = 0x03,
UUID = 0x04,
MD5 = 0x05,
EncryptedBSONValue = 0x06,
CompressedBSONColumn = 0x07,
UserDefined = 0x80
};
struct Binary {
s32 length;
Subtype subtype;
u8 data[length];
};
struct String {
u32 length [[hidden]];
char value[length];
} [[sealed, format("format_string")]];
struct CString {
char value[];
} [[sealed, format("format_string")]];
fn format_string(auto string) {
return string.value;
};
struct ObjectId {
type::time32_t timestamp;
u8 randomValue[5];
u24 counter;
};
struct DBPointer {
String name;
ObjectId value;
};
using Document;
struct Element {
Type type;
CString name;
if (type == Type::Double) {
double value;
} else if (type == Type::String) {
String value;
} else if (type == Type::EmbeddedDocument) {
Document value;
} else if (type == Type::Array) {
Document value;
} else if (type == Type::Binary) {
Binary value;
} else if (type == Type::Undefined) {
/* undefined */
} else if (type == Type::ObjectId) {
ObjectId value;
} else if (type == Type::Boolean) {
bool value;
} else if (type == Type::UTCDatetime) {
type::time64_t value;
} else if (type == Type::Null) {
/* null */
} else if (type == Type::Regex) {
CString regexPattern;
CString regexOptions;
} else if (type == Type::DBPointer) {
DBPointer value;
} else if (type == Type::JavaScript) {
String value;
} else if (type == Type::Symbol) {
String value;
} else if (type == Type::JavaScriptWithScope) {
String value;
} else if (type == Type::Int32) {
s32 value;
} else if (type == Type::Timestamp) {
u64 value;
} else if (type == Type::Int64) {
s64 value;
} else if (type == Type::Decimal128) {
u128 value;
}
};
struct Document {
s32 listLength;
Element elements[while($ < ((addressof(this) + listLength) - 1))];
padding[1];
};
Document documents[while(!std::mem::eof())] @ 0x00 [[inline]];

210
patterns/bsp_goldsrc.hexpat Normal file
View File

@@ -0,0 +1,210 @@
#pragma description GoldSrc engine maps format (used in Half-Life 1)
import std.ptr;
import std.mem;
import std.sys;
#pragma endian little
#define HEADER_LUMPS 15
#define MAX_MAP_HULLS 4
#define NUM_AMBIENTS 4
#define MAXLIGHTMAPS 4
enum LumpIndex : u32 {
Entities,
Planes,
Textures,
Vertexes,
Visibility,
Nodes,
Texinfo,
Faces,
Lighting,
Clipnodes,
Leafs,
Marksurfaces,
Edges,
Surfedges,
Models,
};
struct vec3f
{
float x, y, z;
};
struct dlump_t
{
s32 fileofs;
s32 filelen;
};
struct dheader_t
{
s32 version;
dlump_t lumps[HEADER_LUMPS];
std::assert(version == 30, "This version of BSP format is unsupported.");
};
struct dmodel_t
{
vec3f mins;
vec3f maxs;
vec3f origin; // for sounds or lights
s32 headnode[MAX_MAP_HULLS];
s32 visleafs; // not including the solid leaf 0
s32 firstface;
s32 numfaces;
};
struct dplane_t
{
vec3f normal;
float dist;
s32 type;
};
struct dtexinfo_t
{
float vecs[8]; // texmatrix [s/t][xyz offset]
s32 miptex;
s16 flags;
s16 faceinfo; // -1 no face info otherwise dfaceinfo_t
};
struct dleaf_t
{
s32 contents;
s32 visofs; // -1 = no visibility info
s16 mins[3];
s16 maxs[3];
u16 firstmarksurface;
u16 nummarksurfaces;
u8 ambient_level[NUM_AMBIENTS];
};
struct dnode_t
{
s32 planenum;
s16 children[2]; // negative numbers are -(leafs + 1), not nodes
s16 mins[3];
s16 maxs[3];
u16 firstface;
u16 numfaces; // counting both sides
};
struct dface_t
{
u16 planenum;
s16 side;
s32 firstedge;
s16 numedges;
s16 texinfo;
u8 styles[MAXLIGHTMAPS];
s32 lightofs; // start of [numstyles*surfsize] samples
};
struct dedge_t
{
u16 v[2]; // indices of vertexes
};
struct dclipnode_t
{
s32 planenum;
s16 children[2];
};
using dmarkface_t = u16;
using dsurfedge_t = s32;
using dvertex_t = vec3f;
fn get_miptex_pixeldata_size(auto width, auto height, auto offset) {
if (offset != 0) {
return std::mem::align_to(4, width * height * 85 / 64 + sizeof(u16) + 768);
}
else {
return 0;
}
};
struct miptex_t
{
char name[16];
u32 width;
u32 height;
u32 offsets[4]; // four mip maps stored
u8 pixeldata[get_miptex_pixeldata_size(width, height, offsets[0])];
};
dheader_t file_header @ 0x00;
fn get_lump_element_count(auto index, auto elem_size) {
return file_header.lumps[index].filelen / elem_size;
};
fn get_lump_address(auto index) {
return file_header.lumps[index].fileofs;
};
fn get_miptex_ptr_base(auto offset) {
return file_header.lumps[LumpIndex::Textures].fileofs;
};
struct MiptexPointer
{
miptex_t *data: u32 [[pointer_base("get_miptex_ptr_base"), inline]];
};
struct dmiptexlump_t
{
s32 nummiptex;
MiptexPointer dataofs[nummiptex];
};
struct VisibilityData
{
u8 data[file_header.lumps[LumpIndex::Visibility].filelen];
u8 pad[std::mem::align_to(4, sizeof(this)) - sizeof(this)];
};
struct LightmapData
{
u8 data[file_header.lumps[LumpIndex::Lighting].filelen];
u8 pad[std::mem::align_to(4, sizeof(this)) - sizeof(this)];
};
struct EntityData
{
char text[];
u8 pad[std::mem::align_to(4, sizeof(this)) - sizeof(this)];
};
s32 models_count = get_lump_element_count(LumpIndex::Models, sizeof(dmodel_t));
s32 vertexes_count = get_lump_element_count(LumpIndex::Vertexes, sizeof(vec3f));
s32 planes_count = get_lump_element_count(LumpIndex::Planes, sizeof(dplane_t));
s32 leafs_count = get_lump_element_count(LumpIndex::Leafs, sizeof(dleaf_t));
s32 nodes_count = get_lump_element_count(LumpIndex::Nodes, sizeof(dnode_t));
s32 faces_count = get_lump_element_count(LumpIndex::Faces, sizeof(dface_t));
s32 markfaces_count = get_lump_element_count(LumpIndex::Marksurfaces, sizeof(dmarkface_t));
s32 surfedges_count = get_lump_element_count(LumpIndex::Surfedges, sizeof(dsurfedge_t));
s32 edges_count = get_lump_element_count(LumpIndex::Edges, sizeof(dedge_t));
s32 clipnodes_count = get_lump_element_count(LumpIndex::Clipnodes, sizeof(dclipnode_t));
s32 texinfo_count = get_lump_element_count(LumpIndex::Texinfo, sizeof(dtexinfo_t));
dmodel_t models_lump[models_count] @ get_lump_address(LumpIndex::Models);
dvertex_t vertexes_lump[vertexes_count] @ get_lump_address(LumpIndex::Vertexes);
dplane_t planes_lump[planes_count] @ get_lump_address(LumpIndex::Planes);
dleaf_t leafs_lump[leafs_count] @ get_lump_address(LumpIndex::Leafs);
dnode_t nodes_lump[nodes_count] @ get_lump_address(LumpIndex::Nodes);
dface_t faces_lump[faces_count] @ get_lump_address(LumpIndex::Faces);
dmarkface_t markfaces_lump[markfaces_count] @ get_lump_address(LumpIndex::Marksurfaces);
dsurfedge_t surfedges_lump[surfedges_count] @ get_lump_address(LumpIndex::Surfedges);
dedge_t edges_lump[edges_count] @ get_lump_address(LumpIndex::Edges);
dclipnode_t clipnodes_lump[clipnodes_count] @ get_lump_address(LumpIndex::Clipnodes);
dtexinfo_t texinfo_lump[texinfo_count] @ get_lump_address(LumpIndex::Texinfo);
dmiptexlump_t textures_lump @ get_lump_address(LumpIndex::Textures);
VisibilityData visdata_lump @ get_lump_address(LumpIndex::Visibility);
LightmapData lightdata_lump @ get_lump_address(LumpIndex::Lighting);
EntityData entdata_lump @ get_lump_address(LumpIndex::Entities);

View File

@@ -1,3 +1,5 @@
#pragma description Command and Conquer Voxel Animation
// Command and conquer voxel animation format
struct vec4_s {

View File

@@ -1,3 +1,5 @@
#pragma description Command and Conquer Voxel Palette
// Command and conquer palette format
struct Color {

View File

@@ -1,3 +1,5 @@
#pragma description Command and Conquer Voxel Model
// Command and Conquer voxel model format
struct vec4_s {

31
patterns/cda.hexpat Normal file
View File

@@ -0,0 +1,31 @@
#pragma description Compact Disc Audio track
struct Header {
u32 RIFF;
s32 size;
u32 CDDA;
u32 fmt;
u32 lenghtofthechunck;
u16 versionofcdformat;
u16 numberofrange;
u32 identifier;
};
struct DataInfo {
u32 range;
u32 duration;
u8 rangepositionframes;
u8 rangepositionseconds;
u8 rangepositionminutes;
u8 nullbyte;
u8 durationtrackframes;
u8 durationtrackseconds;
u8 durationtrackminutes;
u8 nullbytee;
};
Header header @ 0;
DataInfo data @ 0x1C;

View File

@@ -1,395 +1,398 @@
#include <type/magic.pat>
#include <type/size.pat>
#include <type/guid.pat>
#include <type/leb128.pat>
#include <std/sys.pat>
enum WindowsLanguageId : u32 {
Arabic_SaudiArabia = 0x401,
Arabic_Iraq = 0x801,
Arabic_Egypt = 0xc01,
Arabic_Libya = 0x1001,
Arabic_Algeria = 0x1401,
Arabic_Morocco = 0x1801,
Arabic_Tunisia = 0x1c01,
Arabic_Oman = 0x2001,
Arabic_Yemen = 0x2401,
Arabic_Syria = 0x2801,
Arabic_Jordan = 0x2c01,
Arabic_Lebanon = 0x3001,
Arabic_Kuwait = 0x3401,
Arabic_UAE = 0x3801,
Arabic_Bahrain = 0x3c01,
Arabic_Qatar = 0x4001,
Bulgarian = 0x402,
Catalan = 0x403,
Valencian = 0x803,
Chinese_Taiwan = 0x404,
Chinese_PRC = 0x804,
Chinese_HongKongSAR = 0xc04,
Chinese_Singapore = 0x1004,
Chinese_MacaoSAR = 0x1404,
Czech = 0x405,
Danish = 0x406,
German_Germany = 0x407,
German_Switzerland = 0x807,
German_Austria = 0xc07,
German_Luxembourg = 0x1007,
German_Liechtenstein = 0x1407,
Greek = 0x408,
English_UnitedStates = 0x409,
English_UnitedKingdom = 0x809,
English_Australia = 0xc09,
English_Canada = 0x1009,
English_NewZealand = 0x1409,
English_Ireland = 0x1809,
English_SouthAfrica = 0x1c09,
English_Jamaica = 0x2009,
English_Caribbean = 0x2409,
English_Belize = 0x2809,
English_TrinidadandTobago = 0x2c09,
English_Zimbabwe = 0x3009,
English_Philippines = 0x3409,
English_Indonesia = 0x3809,
English_HongKongSAR = 0x3c09,
English_India = 0x4009,
English_Malaysia = 0x4409,
English_Singapore = 0x4809,
Spanish_SpainTraditionalSort = 0x40a,
Spanish_Mexico = 0x80a,
Spanish_Spain = 0xc0a,
Spanish_Guatemala = 0x100a,
Spanish_CostaRica = 0x140a,
Spanish_Panama = 0x180a,
Spanish_DominicanRepublic = 0x1c0a,
Spanish_Venezuela = 0x200a,
Spanish_Colombia = 0x240a,
Spanish_Peru = 0x280a,
Spanish_Argentina = 0x2c0a,
Spanish_Ecuador = 0x300a,
Spanish_Chile = 0x340a,
Spanish_Uruguay = 0x380a,
Spanish_Paraguay = 0x3c0a,
Spanish_Bolivia = 0x400a,
Spanish_ElSalvador = 0x440a,
Spanish_Honduras = 0x480a,
Spanish_Nicaragua = 0x4c0a,
Spanish_PuertoRico = 0x500a,
Spanish_UnitedStates = 0x540a,
Spanish_LatinAmerica = 0x580a,
Finnish = 0x40b,
French_France = 0x40c,
French_Belgium = 0x80c,
French_Canada = 0xc0c,
French_Switzerland = 0x100c,
French_Luxembourg = 0x140c,
French_Monaco = 0x180c,
French_Caribbean = 0x1c0c,
French_Reunion = 0x200c,
French_CongoDRC = 0x240c,
French_Senegal = 0x280c,
French_Cameroon = 0x2c0c,
French_CoteDIvoire = 0x300c,
French_Mali = 0x340c,
French_Morocco = 0x380c,
French_Haiti = 0x3c0c,
Hebrew = 0x40d,
Hungarian = 0x40e,
Icelandic = 0x40f,
Italian_Italy = 0x410,
Italian_Switzerland = 0x810,
Japanese = 0x411,
Korean = 0x412,
Dutch_Netherlands = 0x413,
Dutch_Belgium = 0x813,
Norwegian_Bokmal = 0x414,
Norwegian_Nynorsk = 0x814,
Polish = 0x415,
Portuguese_Brazil = 0x416,
Portuguese_Portugal = 0x816,
Romansh = 0x417,
Romanian = 0x418,
Romanian_Moldova = 0x818,
Russian = 0x419,
Russian_Moldova = 0x819,
Croatian_Croatia = 0x41a,
Serbian_LatinSerbiaandMontenegroFormer = 0x81a,
Serbian_CyrillicSerbiaAndMontenegroFormer = 0xc1a,
Croatian_BosniaAndHerzegovina = 0x101a,
Bosnian_Latin = 0x141a,
Serbian_LatinBosniaAndHerzegovina = 0x181a,
Serbian_CyrillicBosniaAndHerzegovina = 0x1c1a,
Bosnian_Cyrillic = 0x201a,
Serbian_LatinSerbia = 0x241a,
Serbian_CyrillicSerbia = 0x281a,
Serbian_LatinMontenegro = 0x2c1a,
Serbian_CyrillicMontenegro = 0x301a,
Slovak = 0x41b,
Albanian = 0x41c,
Swedish_Sweden = 0x41d,
Swedish_Finland = 0x81d,
Thai = 0x41e,
Turkish = 0x41f,
Urdu_Pakistan = 0x420,
Urdu_India = 0x820,
Indonesian = 0x421,
Ukrainian = 0x422,
Belarusian = 0x423,
Slovenian = 0x424,
Estonian = 0x425,
Latvian = 0x426,
Lithuanian = 0x427,
Tajik = 0x428,
Persian = 0x429,
Vietnamese = 0x42a,
Armenian = 0x42b,
Azerbaijani_Latin = 0x42c,
Azerbaijani_Cyrillic = 0x82c,
Basque = 0x42d,
UpperSorbian = 0x42e,
LowerSorbian = 0x82e,
Macedonian = 0x42f,
Sesotho_SouthAfrica = 0x430,
Xitsonga = 0x431,
Setswana_SouthAfrica = 0x432,
Setswana_Botswana = 0x832,
Venda = 0x433,
isiXhosa = 0x434,
isiZulu = 0x435,
Afrikaans = 0x436,
Georgian = 0x437,
Faroese = 0x438,
Hindi = 0x439,
Maltese = 0x43a,
NorthernSami_Norway = 0x43b,
NorthernSami_Sweden = 0x83b,
NorthernSami_Finland = 0xc3b,
LuleSami_Norway = 0x103b,
LuleSami_Sweden = 0x143b,
SouthernSami_Norway = 0x183b,
SouthernSami_Sweden = 0x1c3b,
SkoltSami_Finland = 0x203b,
InariSami_Finland = 0x243b,
Irish = 0x83c,
Yiddish = 0x43d,
Malay_Malaysia = 0x43e,
Malay_BruneiDarussalam = 0x83e,
Kazakh = 0x43f,
Kyrgyz = 0x440,
Kiswahili = 0x441,
Turkmen = 0x442,
Uzbek_Latin = 0x443,
Uzbek_Cyrillic = 0x843,
Tatar = 0x444,
Bangla_India = 0x445,
Bangla_Bangladesh = 0x845,
Punjabi_India = 0x446,
Punjabi_Pakistan = 0x846,
Gujarati = 0x447,
Odia = 0x448,
Tamil_India = 0x449,
Tamil_SriLanka = 0x849,
Telugu = 0x44a,
Kannada = 0x44b,
Malayalam = 0x44c,
Assamese = 0x44d,
Marathi = 0x44e,
Sanskrit = 0x44f,
Mongolian_Cyrillic = 0x450,
Mongolian_TraditionalMongolianPRC = 0x850,
Mongolian_TraditionalMongolianMongolia = 0xc50,
Tibetan_PRC = 0x451,
Welsh = 0x452,
Khmer = 0x453,
Lao = 0x454,
Burmese = 0x455,
Galician = 0x456,
Konkani = 0x457,
Manipuri = 0x458,
Sindhi_Devanagari = 0x459,
Sindhi_Arabic = 0x859,
Syriac = 0x45a,
Sinhala = 0x45b,
Cherokee_Cherokee = 0x45c,
Inuktitut_Syllabics = 0x45d,
Inuktitut_Latin = 0x85d,
Amharic = 0x45e,
Tamazight_ArabicMorocco = 0x45f,
Tamazight_LatinAlgeria = 0x85f,
Tamazight_TifinaghMorocco = 0x105f,
Kashmiri_Arabic = 0x460,
Kashmiri = 0x860,
Nepali = 0x461,
Nepali_India = 0x861,
Frisian = 0x462,
Pashto = 0x463,
Filipino = 0x464,
Divehi = 0x465,
Edo = 0x466,
Fulah_Nigeria = 0x467,
Fulah_LatinSenegal = 0x867,
Hausa = 0x468,
Ibibio_Nigeria = 0x469,
Yoruba = 0x46a,
Quechua_Bolivia = 0x46b,
Quechua_Ecuador = 0x86b,
Quechua_Peru = 0xc6b,
SesothoSaLeboa = 0x46c,
Bashkir = 0x46d,
Luxembourgish = 0x46e,
Greenlandic = 0x46f,
Igbo = 0x470,
Kanuri = 0x471,
Oromo = 0x472,
Tigrinya_Ethiopia = 0x473,
Tigrinya_Eritrea = 0x873,
Guarani = 0x474,
Hawaiian = 0x475,
Latin = 0x476,
Somali = 0x477,
Yi_PRC = 0x478,
Papiamentu = 0x479,
Mapudungun = 0x47a,
Mohawk = 0x47c,
Breton = 0x47e,
Uyghur_PRC = 0x480,
Maori = 0x481,
Occitan = 0x482,
Corsican = 0x483,
Alsatian = 0x484,
Sakha = 0x485,
Kiche = 0x486,
Kinyarwanda = 0x487,
Wolof = 0x488,
Dari = 0x48c,
ScottishGaelic_UnitedKingdom = 0x491,
CentralKurdish_Iraq = 0x492
};
struct DirectoryListingEntry {
type::LEB128 nameLength;
char name[nameLength];
type::LEB128 contentSection;
type::LEB128 offset;
type::LEB128 length;
};
struct DirectoryIndexEntry {
type::LEB128 nameLength;
char name[nameLength];
type::LEB128 directoryListingChunk;
};
struct ListingChunk {
char magic[4];
if (magic == "PMGL") {
type::Size<u32> freeSpaceLength;
u32;
u32 prevChunkNumber, nextChunkNumber;
u16 directoryListingEntryCount @ addressof(this) + parent.directoryChunkSize - 2;
u16 offsets[(freeSpaceLength - 2) / 2] @ addressof(directoryListingEntryCount) - (freeSpaceLength - 2);
DirectoryListingEntry directories[directoryListingEntryCount];
$ = addressof(directoryListingEntryCount) + sizeof(directoryListingEntryCount);
} else if (magic == "PMGI") {
type::Size<u32> freeSpaceLength;
u16 directoryIndexEntryCount @ addressof(this) + parent.directoryChunkSize - 2;
u16 offsets[(freeSpaceLength - 2) / 2] @ addressof(directoryIndexEntryCount) - (freeSpaceLength - 2);
DirectoryIndexEntry indexes[directoryIndexEntryCount];
$ = addressof(directoryIndexEntryCount) + sizeof(directoryIndexEntryCount);
} else {
std::error("Invalid chunk magic!");
}
};
struct HeaderSection {
char magic[4];
if (magic == "\xFE\x01\x00\x00") {
u32;
type::Size<u64> fileSize;
u32;
u32;
} else if (magic == "ITSP") {
u32 version;
type::Size<u32> directoryHeaderLength1;
u32;
u32 directoryChunkSize;
u32 quickRefSectionDensity;
u32 indexTreeDepth;
u32 rootIndexChunkNumber;
u32 firstPMGLChunkNumber;
u32 lastPMGLChunkNumber;
u32;
u32 directoryChunkCount;
WindowsLanguageId languageId;
type::GUID guid;
type::Size<u32> directoryHeaderLength2;
u32;
u32;
u32;
ListingChunk chunk[directoryChunkCount];
} else {
std::error("Invalid header section magic!");
}
};
struct HeaderSectionTableEntry {
u64 offset;
type::Size<u64> size;
HeaderSection headerSection @ offset;
};
struct NameListEntry {
type::Size<u16> nameLength;
char16 name[nameLength];
padding[2];
};
struct NameListFile {
u16 fileLengthWords;
u16 entriesInFile;
NameListEntry nameList[entriesInFile];
padding[0x2E];
};
struct SectionData {
u32 fileLengthWords;
type::Magic<"LZXC"> magic;
u32 version;
u32 lzxResetInterval;
type::Size<u32> windowSize;
type::Size<u32> cacheSize;
u32;
};
struct Content {
NameListFile nameListFile;
SectionData sectionData;
};
struct CHM {
type::Magic<"ITSF"> magic;
u32 version;
type::Size<u32> headerSize;
u32;
be u32 timeStamp;
WindowsLanguageId languageId;
type::GUID guids[2];
HeaderSectionTableEntry headerSectionTable[2];
Content *dataOffset : u64;
};
CHM chm @ 0x00;
#pragma author WerWolv
#pragma description Windows HtmlHelp Data (ITSF / CHM)
import type.magic;
import type.size;
import type.guid;
import type.leb128;
import std.sys;
enum WindowsLanguageId : u32 {
Arabic_SaudiArabia = 0x401,
Arabic_Iraq = 0x801,
Arabic_Egypt = 0xc01,
Arabic_Libya = 0x1001,
Arabic_Algeria = 0x1401,
Arabic_Morocco = 0x1801,
Arabic_Tunisia = 0x1c01,
Arabic_Oman = 0x2001,
Arabic_Yemen = 0x2401,
Arabic_Syria = 0x2801,
Arabic_Jordan = 0x2c01,
Arabic_Lebanon = 0x3001,
Arabic_Kuwait = 0x3401,
Arabic_UAE = 0x3801,
Arabic_Bahrain = 0x3c01,
Arabic_Qatar = 0x4001,
Bulgarian = 0x402,
Catalan = 0x403,
Valencian = 0x803,
Chinese_Taiwan = 0x404,
Chinese_PRC = 0x804,
Chinese_HongKongSAR = 0xc04,
Chinese_Singapore = 0x1004,
Chinese_MacaoSAR = 0x1404,
Czech = 0x405,
Danish = 0x406,
German_Germany = 0x407,
German_Switzerland = 0x807,
German_Austria = 0xc07,
German_Luxembourg = 0x1007,
German_Liechtenstein = 0x1407,
Greek = 0x408,
English_UnitedStates = 0x409,
English_UnitedKingdom = 0x809,
English_Australia = 0xc09,
English_Canada = 0x1009,
English_NewZealand = 0x1409,
English_Ireland = 0x1809,
English_SouthAfrica = 0x1c09,
English_Jamaica = 0x2009,
English_Caribbean = 0x2409,
English_Belize = 0x2809,
English_TrinidadandTobago = 0x2c09,
English_Zimbabwe = 0x3009,
English_Philippines = 0x3409,
English_Indonesia = 0x3809,
English_HongKongSAR = 0x3c09,
English_India = 0x4009,
English_Malaysia = 0x4409,
English_Singapore = 0x4809,
Spanish_SpainTraditionalSort = 0x40a,
Spanish_Mexico = 0x80a,
Spanish_Spain = 0xc0a,
Spanish_Guatemala = 0x100a,
Spanish_CostaRica = 0x140a,
Spanish_Panama = 0x180a,
Spanish_DominicanRepublic = 0x1c0a,
Spanish_Venezuela = 0x200a,
Spanish_Colombia = 0x240a,
Spanish_Peru = 0x280a,
Spanish_Argentina = 0x2c0a,
Spanish_Ecuador = 0x300a,
Spanish_Chile = 0x340a,
Spanish_Uruguay = 0x380a,
Spanish_Paraguay = 0x3c0a,
Spanish_Bolivia = 0x400a,
Spanish_ElSalvador = 0x440a,
Spanish_Honduras = 0x480a,
Spanish_Nicaragua = 0x4c0a,
Spanish_PuertoRico = 0x500a,
Spanish_UnitedStates = 0x540a,
Spanish_LatinAmerica = 0x580a,
Finnish = 0x40b,
French_France = 0x40c,
French_Belgium = 0x80c,
French_Canada = 0xc0c,
French_Switzerland = 0x100c,
French_Luxembourg = 0x140c,
French_Monaco = 0x180c,
French_Caribbean = 0x1c0c,
French_Reunion = 0x200c,
French_CongoDRC = 0x240c,
French_Senegal = 0x280c,
French_Cameroon = 0x2c0c,
French_CoteDIvoire = 0x300c,
French_Mali = 0x340c,
French_Morocco = 0x380c,
French_Haiti = 0x3c0c,
Hebrew = 0x40d,
Hungarian = 0x40e,
Icelandic = 0x40f,
Italian_Italy = 0x410,
Italian_Switzerland = 0x810,
Japanese = 0x411,
Korean = 0x412,
Dutch_Netherlands = 0x413,
Dutch_Belgium = 0x813,
Norwegian_Bokmal = 0x414,
Norwegian_Nynorsk = 0x814,
Polish = 0x415,
Portuguese_Brazil = 0x416,
Portuguese_Portugal = 0x816,
Romansh = 0x417,
Romanian = 0x418,
Romanian_Moldova = 0x818,
Russian = 0x419,
Russian_Moldova = 0x819,
Croatian_Croatia = 0x41a,
Serbian_LatinSerbiaandMontenegroFormer = 0x81a,
Serbian_CyrillicSerbiaAndMontenegroFormer = 0xc1a,
Croatian_BosniaAndHerzegovina = 0x101a,
Bosnian_Latin = 0x141a,
Serbian_LatinBosniaAndHerzegovina = 0x181a,
Serbian_CyrillicBosniaAndHerzegovina = 0x1c1a,
Bosnian_Cyrillic = 0x201a,
Serbian_LatinSerbia = 0x241a,
Serbian_CyrillicSerbia = 0x281a,
Serbian_LatinMontenegro = 0x2c1a,
Serbian_CyrillicMontenegro = 0x301a,
Slovak = 0x41b,
Albanian = 0x41c,
Swedish_Sweden = 0x41d,
Swedish_Finland = 0x81d,
Thai = 0x41e,
Turkish = 0x41f,
Urdu_Pakistan = 0x420,
Urdu_India = 0x820,
Indonesian = 0x421,
Ukrainian = 0x422,
Belarusian = 0x423,
Slovenian = 0x424,
Estonian = 0x425,
Latvian = 0x426,
Lithuanian = 0x427,
Tajik = 0x428,
Persian = 0x429,
Vietnamese = 0x42a,
Armenian = 0x42b,
Azerbaijani_Latin = 0x42c,
Azerbaijani_Cyrillic = 0x82c,
Basque = 0x42d,
UpperSorbian = 0x42e,
LowerSorbian = 0x82e,
Macedonian = 0x42f,
Sesotho_SouthAfrica = 0x430,
Xitsonga = 0x431,
Setswana_SouthAfrica = 0x432,
Setswana_Botswana = 0x832,
Venda = 0x433,
isiXhosa = 0x434,
isiZulu = 0x435,
Afrikaans = 0x436,
Georgian = 0x437,
Faroese = 0x438,
Hindi = 0x439,
Maltese = 0x43a,
NorthernSami_Norway = 0x43b,
NorthernSami_Sweden = 0x83b,
NorthernSami_Finland = 0xc3b,
LuleSami_Norway = 0x103b,
LuleSami_Sweden = 0x143b,
SouthernSami_Norway = 0x183b,
SouthernSami_Sweden = 0x1c3b,
SkoltSami_Finland = 0x203b,
InariSami_Finland = 0x243b,
Irish = 0x83c,
Yiddish = 0x43d,
Malay_Malaysia = 0x43e,
Malay_BruneiDarussalam = 0x83e,
Kazakh = 0x43f,
Kyrgyz = 0x440,
Kiswahili = 0x441,
Turkmen = 0x442,
Uzbek_Latin = 0x443,
Uzbek_Cyrillic = 0x843,
Tatar = 0x444,
Bangla_India = 0x445,
Bangla_Bangladesh = 0x845,
Punjabi_India = 0x446,
Punjabi_Pakistan = 0x846,
Gujarati = 0x447,
Odia = 0x448,
Tamil_India = 0x449,
Tamil_SriLanka = 0x849,
Telugu = 0x44a,
Kannada = 0x44b,
Malayalam = 0x44c,
Assamese = 0x44d,
Marathi = 0x44e,
Sanskrit = 0x44f,
Mongolian_Cyrillic = 0x450,
Mongolian_TraditionalMongolianPRC = 0x850,
Mongolian_TraditionalMongolianMongolia = 0xc50,
Tibetan_PRC = 0x451,
Welsh = 0x452,
Khmer = 0x453,
Lao = 0x454,
Burmese = 0x455,
Galician = 0x456,
Konkani = 0x457,
Manipuri = 0x458,
Sindhi_Devanagari = 0x459,
Sindhi_Arabic = 0x859,
Syriac = 0x45a,
Sinhala = 0x45b,
Cherokee_Cherokee = 0x45c,
Inuktitut_Syllabics = 0x45d,
Inuktitut_Latin = 0x85d,
Amharic = 0x45e,
Tamazight_ArabicMorocco = 0x45f,
Tamazight_LatinAlgeria = 0x85f,
Tamazight_TifinaghMorocco = 0x105f,
Kashmiri_Arabic = 0x460,
Kashmiri = 0x860,
Nepali = 0x461,
Nepali_India = 0x861,
Frisian = 0x462,
Pashto = 0x463,
Filipino = 0x464,
Divehi = 0x465,
Edo = 0x466,
Fulah_Nigeria = 0x467,
Fulah_LatinSenegal = 0x867,
Hausa = 0x468,
Ibibio_Nigeria = 0x469,
Yoruba = 0x46a,
Quechua_Bolivia = 0x46b,
Quechua_Ecuador = 0x86b,
Quechua_Peru = 0xc6b,
SesothoSaLeboa = 0x46c,
Bashkir = 0x46d,
Luxembourgish = 0x46e,
Greenlandic = 0x46f,
Igbo = 0x470,
Kanuri = 0x471,
Oromo = 0x472,
Tigrinya_Ethiopia = 0x473,
Tigrinya_Eritrea = 0x873,
Guarani = 0x474,
Hawaiian = 0x475,
Latin = 0x476,
Somali = 0x477,
Yi_PRC = 0x478,
Papiamentu = 0x479,
Mapudungun = 0x47a,
Mohawk = 0x47c,
Breton = 0x47e,
Uyghur_PRC = 0x480,
Maori = 0x481,
Occitan = 0x482,
Corsican = 0x483,
Alsatian = 0x484,
Sakha = 0x485,
Kiche = 0x486,
Kinyarwanda = 0x487,
Wolof = 0x488,
Dari = 0x48c,
ScottishGaelic_UnitedKingdom = 0x491,
CentralKurdish_Iraq = 0x492
};
struct DirectoryListingEntry {
type::LEB128 nameLength;
char name[nameLength];
type::LEB128 contentSection;
type::LEB128 offset;
type::LEB128 length;
};
struct DirectoryIndexEntry {
type::LEB128 nameLength;
char name[nameLength];
type::LEB128 directoryListingChunk;
};
struct ListingChunk {
char magic[4];
if (magic == "PMGL") {
type::Size<u32> freeSpaceLength;
u32;
u32 prevChunkNumber, nextChunkNumber;
u16 directoryListingEntryCount @ addressof(this) + parent.directoryChunkSize - 2;
u16 offsets[(freeSpaceLength - 2) / 2] @ addressof(directoryListingEntryCount) - (freeSpaceLength - 2);
DirectoryListingEntry directories[directoryListingEntryCount];
$ = addressof(directoryListingEntryCount) + sizeof(directoryListingEntryCount);
} else if (magic == "PMGI") {
type::Size<u32> freeSpaceLength;
u16 directoryIndexEntryCount @ addressof(this) + parent.directoryChunkSize - 2;
u16 offsets[(freeSpaceLength - 2) / 2] @ addressof(directoryIndexEntryCount) - (freeSpaceLength - 2);
DirectoryIndexEntry indexes[directoryIndexEntryCount];
$ = addressof(directoryIndexEntryCount) + sizeof(directoryIndexEntryCount);
} else {
std::error("Invalid chunk magic!");
}
};
struct HeaderSection {
char magic[4];
if (magic == "\xFE\x01\x00\x00") {
u32;
type::Size<u64> fileSize;
u32;
u32;
} else if (magic == "ITSP") {
u32 version;
type::Size<u32> directoryHeaderLength1;
u32;
u32 directoryChunkSize;
u32 quickRefSectionDensity;
u32 indexTreeDepth;
u32 rootIndexChunkNumber;
u32 firstPMGLChunkNumber;
u32 lastPMGLChunkNumber;
u32;
u32 directoryChunkCount;
WindowsLanguageId languageId;
type::GUID guid;
type::Size<u32> directoryHeaderLength2;
u32;
u32;
u32;
ListingChunk chunk[directoryChunkCount];
} else {
std::error("Invalid header section magic!");
}
};
struct HeaderSectionTableEntry {
u64 offset;
type::Size<u64> size;
HeaderSection headerSection @ offset;
};
struct NameListEntry {
type::Size<u16> nameLength;
char16 name[nameLength];
padding[2];
};
struct NameListFile {
u16 fileLengthWords;
u16 entriesInFile;
NameListEntry nameList[entriesInFile];
padding[0x2E];
};
struct SectionData {
u32 fileLengthWords;
type::Magic<"LZXC"> magic;
u32 version;
u32 lzxResetInterval;
type::Size<u32> windowSize;
type::Size<u32> cacheSize;
u32;
};
struct Content {
NameListFile nameListFile;
SectionData sectionData;
};
struct CHM {
type::Magic<"ITSF"> magic;
u32 version;
type::Size<u32> headerSize;
u32;
be u32 timeStamp;
WindowsLanguageId languageId;
type::GUID guids[2];
HeaderSectionTableEntry headerSectionTable[2];
Content *dataOffset : u64;
};
CHM chm @ 0x00;

View File

@@ -1,209 +1,212 @@
#pragma MIME application/x-coff
#include <type/time.pat>
#include <type/size.pat>
enum Machine : u16 {
Unknown = 0x0000,
AM33 = 0x01D3,
AMD64 = 0x8664,
ARM = 0x01C0,
ARM64 = 0xAA64,
ARMNT = 0x01C4,
EBC = 0x0EBC,
I386 = 0x014C,
IA64 = 0x0200,
LOONGARCH32 = 0x6232,
LOONGARCH64 = 0x6264,
M32R = 0x9041,
MIPS16 = 0x0226,
MIPSFPU = 0x0366,
MIPSFPU16 = 0x0466,
POWERPC = 0x01F0,
POWERPCFP = 0x01F0,
R4000 = 0x0166,
RISCV32 = 0x5032,
RISCV64 = 0x5064,
RISCV128 = 0x5128,
SH3 = 0x01A2,
SH3DSP = 0x01A3,
SH4 = 0x01A6,
SH5 = 0x01A8,
THUMB = 0x01C2,
WCEMIPSV2 = 0x0169
};
bitfield Characteristics {
relocsStripped : 1;
executableImage : 1;
lineNumsStripped : 1;
localSymsStripped : 1;
aggressiveWsTrim : 1;
largeAddressAware : 1;
padding : 1;
bytesReversedLo : 1;
_32BitMachine : 1;
debugStripped : 1;
removableRunFromSwap : 1;
netRunFromSwap : 1;
system : 1;
dll : 1;
upSystemOnly : 1;
bytesReversedHi : 1;
} [[right_to_left]];
enum Type : u16 {
Null = 0,
Void = 1,
Char = 2,
Short = 3,
Int = 4,
Long = 5,
Float = 6,
Double = 7,
Struct = 8,
Union = 9,
Enum = 10,
MOE = 11,
Byte = 12,
Word = 13,
UInt = 14,
DWord = 15
};
enum StorageClass : u8 {
EndOfFunction = 0xFF,
Null = 0,
Automatic = 1,
External = 2,
Static = 3,
Register = 4,
ExternalDef = 5,
Label = 6,
UndefinedLabel = 7,
MemberOfStruct = 8,
Argument = 9,
StructTag = 10,
MemberOfUnion = 11,
UnionTag = 12,
TypeDefinition = 13,
UndefinedStatic = 14,
EnumTag = 15,
MemberOfEnum = 16,
RegisterParam = 17,
BitField = 18,
Block = 100,
Function = 101,
EndOfStruct = 102,
File = 103,
Section = 104,
WeakExternal = 105,
CLRToken = 107
};
struct AuxSymbol {
u8 data[18];
};
u32 countedSymbols = 0;
struct SymbolTable {
char name[8];
u32 value;
u16 sectionNumber;
Type type;
StorageClass storageClass;
u8 numberOfAuxSymbols;
countedSymbols += 1 + numberOfAuxSymbols;
AuxSymbol auxSymbols[numberOfAuxSymbols];
if (countedSymbols >= parent.header.numberOfSymbols)
break;
};
struct String {
char value[];
};
struct StringTable {
u32 size;
String strings[while($ < addressof(size) + size)];
};
bitfield SectionFlags {
padding : 3;
typeNoPad : 1;
padding : 1;
cntCode : 1;
initializedData : 1;
uninitializedData : 1;
lnkOther : 1;
lnkInfo : 1;
padding : 1;
lnkRemove : 1;
lnkCOMDAT : 1;
padding : 2;
gprel : 1;
padding : 1;
memPurgeable : 1;
memLocked : 1;
memPreload : 1;
alignment : 4 [[format("format_alignment")]];
lnkNrelocOvfl : 1;
memDiscardable : 1;
memNotCached : 1;
memNotPaged : 1;
memShared : 1;
memExecute : 1;
memRead : 1;
memWrite : 1;
} [[right_to_left]];
fn format_alignment(u8 alignment) {
return 1 << alignment;
};
struct Relocations {
u32 virtualAddress;
u32 symbolTableIndex;
Type type;
};
struct Section {
char name[8];
type::Size<u32> virtualSize;
u32 virtualAddress;
type::Size<u32> sizeOfRawData;
u32 pointerToRawData;
u32 pointerToRelocations;
u32 pointerToLineNumbers;
u16 numberOfRelocations;
u16 numberOfLineNumbers;
SectionFlags characteristics;
u8 rawData[sizeOfRawData] @ pointerToRawData [[sealed]];
Relocations relocations[numberOfRelocations] @ pointerToRelocations;
};
struct Header {
Machine machine;
u16 numberOfSections;
type::time32_t timeDateStamp;
u32 pointerToSymbolTable;
u32 numberOfSymbols;
u16 sizeOfOptionalHeader;
Characteristics characteristics;
};
struct COFF {
Header header;
Section sectionTable[header.numberOfSections];
SymbolTable symbolTable[header.numberOfSymbols] @ header.pointerToSymbolTable;
StringTable stringTable @ addressof(symbolTable) + sizeof(symbolTable);
};
COFF coff @ 0x00;
#pragma author WerWolv
#pragma description Common Object File Format (COFF) executable
#pragma MIME application/x-coff
import type.time;
import type.size;
enum Machine : u16 {
Unknown = 0x0000,
AM33 = 0x01D3,
AMD64 = 0x8664,
ARM = 0x01C0,
ARM64 = 0xAA64,
ARMNT = 0x01C4,
EBC = 0x0EBC,
I386 = 0x014C,
IA64 = 0x0200,
LOONGARCH32 = 0x6232,
LOONGARCH64 = 0x6264,
M32R = 0x9041,
MIPS16 = 0x0226,
MIPSFPU = 0x0366,
MIPSFPU16 = 0x0466,
POWERPC = 0x01F0,
POWERPCFP = 0x01F0,
R4000 = 0x0166,
RISCV32 = 0x5032,
RISCV64 = 0x5064,
RISCV128 = 0x5128,
SH3 = 0x01A2,
SH3DSP = 0x01A3,
SH4 = 0x01A6,
SH5 = 0x01A8,
THUMB = 0x01C2,
WCEMIPSV2 = 0x0169
};
bitfield Characteristics {
relocsStripped : 1;
executableImage : 1;
lineNumsStripped : 1;
localSymsStripped : 1;
aggressiveWsTrim : 1;
largeAddressAware : 1;
padding : 1;
bytesReversedLo : 1;
_32BitMachine : 1;
debugStripped : 1;
removableRunFromSwap : 1;
netRunFromSwap : 1;
system : 1;
dll : 1;
upSystemOnly : 1;
bytesReversedHi : 1;
};
enum Type : u16 {
Null = 0,
Void = 1,
Char = 2,
Short = 3,
Int = 4,
Long = 5,
Float = 6,
Double = 7,
Struct = 8,
Union = 9,
Enum = 10,
MOE = 11,
Byte = 12,
Word = 13,
UInt = 14,
DWord = 15
};
enum StorageClass : u8 {
EndOfFunction = 0xFF,
Null = 0,
Automatic = 1,
External = 2,
Static = 3,
Register = 4,
ExternalDef = 5,
Label = 6,
UndefinedLabel = 7,
MemberOfStruct = 8,
Argument = 9,
StructTag = 10,
MemberOfUnion = 11,
UnionTag = 12,
TypeDefinition = 13,
UndefinedStatic = 14,
EnumTag = 15,
MemberOfEnum = 16,
RegisterParam = 17,
BitField = 18,
Block = 100,
Function = 101,
EndOfStruct = 102,
File = 103,
Section = 104,
WeakExternal = 105,
CLRToken = 107
};
struct AuxSymbol {
u8 data[18];
};
u32 countedSymbols = 0;
struct SymbolTable {
char name[8];
u32 value;
u16 sectionNumber;
Type type;
StorageClass storageClass;
u8 numberOfAuxSymbols;
countedSymbols += 1 + numberOfAuxSymbols;
AuxSymbol auxSymbols[numberOfAuxSymbols];
if (countedSymbols >= parent.header.numberOfSymbols)
break;
};
struct String {
char value[];
};
struct StringTable {
u32 size;
String strings[while($ < addressof(size) + size)];
};
bitfield SectionFlags {
padding : 3;
typeNoPad : 1;
padding : 1;
cntCode : 1;
initializedData : 1;
uninitializedData : 1;
lnkOther : 1;
lnkInfo : 1;
padding : 1;
lnkRemove : 1;
lnkCOMDAT : 1;
padding : 2;
gprel : 1;
padding : 1;
memPurgeable : 1;
memLocked : 1;
memPreload : 1;
alignment : 4 [[format("format_alignment")]];
lnkNrelocOvfl : 1;
memDiscardable : 1;
memNotCached : 1;
memNotPaged : 1;
memShared : 1;
memExecute : 1;
memRead : 1;
memWrite : 1;
};
fn format_alignment(u8 alignment) {
return 1 << alignment;
};
struct Relocations {
u32 virtualAddress;
u32 symbolTableIndex;
Type type;
};
struct Section {
char name[8];
type::Size<u32> virtualSize;
u32 virtualAddress;
type::Size<u32> sizeOfRawData;
u32 pointerToRawData;
u32 pointerToRelocations;
u32 pointerToLineNumbers;
u16 numberOfRelocations;
u16 numberOfLineNumbers;
SectionFlags characteristics;
u8 rawData[sizeOfRawData] @ pointerToRawData [[sealed]];
Relocations relocations[numberOfRelocations] @ pointerToRelocations;
};
struct Header {
Machine machine;
u16 numberOfSections;
type::time32_t timeDateStamp;
u32 pointerToSymbolTable;
u32 numberOfSymbols;
u16 sizeOfOptionalHeader;
Characteristics characteristics;
};
struct COFF {
Header header;
Section sectionTable[header.numberOfSections];
SymbolTable symbolTable[header.numberOfSymbols] @ header.pointerToSymbolTable;
StringTable stringTable @ addressof(symbolTable) + sizeof(symbolTable);
};
COFF coff @ 0x00;

View File

@@ -1,66 +1,69 @@
#include <type/base.pat>
#include <std/time.pat>
#include <std/core.pat>
#include <std/sys.pat>
#include <std/mem.pat>
#pragma MIME application/x-cpio
namespace old_binary {
using Time = u32 [[format("old_binary::format_time")]];
fn swap_32bit(u32 value) {
return ((value >> 16) & 0xFFFF) | ((value & 0xFFFF) << 16);
};
fn format_time(u32 value) {
return std::time::format(std::time::to_utc(swap_32bit(value)));
};
using SwappedU32 = u32 [[transform("old_binary::swap_32bit"), format("old_binary::swap_32bit")]];
bitfield Mode {
file_type : 4;
suid : 1;
sgid : 1;
sticky : 1;
r : 3;
w : 3;
x : 3;
} [[left_to_right]];
struct CpioHeader {
type::Oct<u16> magic;
if (magic == be u16(0o070707))
std::core::set_endian(std::mem::Endian::Big);
else if (magic == le u16(0o070707))
std::core::set_endian(std::mem::Endian::Little);
else
std::error("Invalid CPIO Magic!");
u16 dev;
u16 ino;
Mode mode;
u16 uid;
u16 gid;
u16 nlink;
u16 rdev;
Time mtime;
u16 namesize;
SwappedU32 filesize;
};
struct Cpio {
CpioHeader header;
char pathname[header.namesize % 2 == 0 ? header.namesize : header.namesize + 1];
u8 data[header.filesize % 2 == 0 ? header.filesize : header.filesize + 1];
if (pathname == "TRAILER!!!\x00\x00")
break;
};
}
old_binary::Cpio cpio[while(true)] @ 0x00;
#pragma author WerWolv
#pragma description Old Binary CPIO Format
import type.base;
import std.time;
import std.core;
import std.sys;
import std.mem;
#pragma MIME application/x-cpio
namespace old_binary {
using Time = u32 [[format("old_binary::format_time")]];
fn swap_32bit(u32 value) {
return ((value >> 16) & 0xFFFF) | ((value & 0xFFFF) << 16);
};
fn format_time(u32 value) {
return std::time::format(std::time::to_utc(swap_32bit(value)));
};
using SwappedU32 = u32 [[transform("old_binary::swap_32bit"), format("old_binary::swap_32bit")]];
bitfield Mode {
x : 3;
w : 3;
r : 3;
sticky : 1;
sgid : 1;
suid : 1;
file_type : 4;
};
struct CpioHeader {
type::Oct<u16> magic;
if (magic == be u16(0o070707))
std::core::set_endian(std::mem::Endian::Big);
else if (magic == le u16(0o070707))
std::core::set_endian(std::mem::Endian::Little);
else
std::error("Invalid CPIO Magic!");
u16 dev;
u16 ino;
Mode mode;
u16 uid;
u16 gid;
u16 nlink;
u16 rdev;
Time mtime;
u16 namesize;
SwappedU32 filesize;
};
struct Cpio {
CpioHeader header;
char pathname[header.namesize % 2 == 0 ? header.namesize : header.namesize + 1];
u8 data[header.filesize % 2 == 0 ? header.filesize : header.filesize + 1];
if (pathname == "TRAILER!!!\x00\x00")
break;
};
}
old_binary::Cpio cpio[while(true)] @ 0x00;

View File

@@ -1,3 +1,5 @@
#pragma description DirectDraw Surface
#pragma MIME image/vnd-ms.dds
#pragma endian little
@@ -165,16 +167,26 @@ bitfield Caps2Flags {
};
bitfield PixelFormatFlags {
alphaPixels : 1;
alpha : 1;
fourCC : 1;
padding : 3;
rgb : 1; // 0x40
padding : 2;
yuv : 1; // 0x200
padding : 3;
luminance : 1; // 0x20000
padding : 17;
alphaPixels : 1; // DDPF_ALPHAPIXELS
alpha : 1; // DDPF_ALPHA
fourCC : 1; // DDPF_FOURCC
paletteIndexed4 : 1; // DDPF_PALETTEINDEXED4
paletteIndexedTo8 : 1; // DDPF_PALETTEINDEXEDTO8
paletteIndexed8 : 1; // DDPF_PALETTEINDEXED8
rgb : 1; // DDPF_RGB
compressed : 1; // DDPF_COMPRESSED
rgbToYuv : 1; // DDPF_RGBTOYUV
yuv : 1; // DDPF_YUV
zBuffer : 1; // DDPF_ZBUFFER
paletteIndexed1 : 1; // DDPF_PALETTEINDEXED1
paletteIndexed2 : 1; // DDPF_PALETTEINDEXED2
zPixels : 1; // DDPF_ZPIXELS
stencilBuffer : 1; // DDPF_STENCILBUFFER
alphaResult : 1; // DDPF_ALPHARESULT
luminance : 1; // DDPF_LUMINANCE
bumpLuminance : 1; // DDPF_BUMPLUMINANCE
bumpDudv : 1; // DDPF_BUMPDUDV
padding : 13; // Padding bits to complete 32-bit structure
};
enum DX10ResourceDimension : u32 {

179
patterns/dex.hexpat Normal file
View File

@@ -0,0 +1,179 @@
#pragma description Dalvik EXecutable Format
import type.leb128;
struct header_item {
u8 magic[8];
u32 checksum;
u8 signature[20];
u32 file_size;
u32 header_size;
u32 endian_tag;
u32 link_size;
u32 link_off;
u32 map_off;
u32 string_ids_size;
u32 string_ids_off;
u32 type_ids_size;
u32 type_ids_off;
u32 proto_ids_size;
u32 proto_ids_off;
u32 field_ids_size;
u32 field_ids_off;
u32 method_ids_size;
u32 method_ids_off;
u32 class_defs_size;
u32 class_defs_off;
u32 data_size;
u32 data_off;
};
struct map_item {
u16 type;
u16 unused;
u32 size;
u32 offset;
};
struct map_list {
u32 size;
map_item list[size];
};
struct string_data_item {
type::uLEB128 utf16_size[[hidden]];
char string[utf16_size];
}[[inline]];
struct string_id_item {
string_data_item* string_data: u32;
}[[name(string_data.string)]];
struct type_id_item {
u32 descriptor_idx;
char type_name[] @ addressof(parent.string_ids[descriptor_idx].string_data.string);
}[[name(type_name)]];
struct proto_id_item {
u32 shorty_idx;
u32 return_type_idx;
u32 parameters_off;
char shorty_dec[] @ addressof(parent.string_ids[shorty_idx].string_data.string);
char return_type[] @ addressof(parent.type_ids[return_type_idx].type_name);
}[[name(shorty_dec)]];
struct field_id_item {
u16 class_idx;
u16 type_idx;
u32 name_idx;
char class_name[] @ addressof(parent.type_ids[class_idx].type_name);
char type_name[] @ addressof(parent.type_ids[type_idx].type_name);
char field_name[] @ addressof(parent.string_ids[name_idx].string_data.string);
}[[name(field_name)]];
struct method_id_item {
u16 class_idx;
u16 proto_idx;
u32 name_idx;
char class_name[] @ addressof(parent.type_ids[class_idx].type_name);
char proto_desc[] @ addressof(parent.proto_ids[proto_idx].shorty_dec);
char method_name[] @ addressof(parent.string_ids[name_idx].string_data.string);
}[[name(class_name+method_name)]];
struct class_site_id_item {
u32 call_site_off;
};
struct method_handle_item {
u16 method_handle_type;
u16 unused;
u16 field_or_method_id;
u16 unused2;
};
enum access_flag : type::uLEB128{
public = 0x1,
private = 0x2,
protected = 0x4,
static = 0x8,
final = 0x10,
synchronized = 0x20,
volatile = 0x40
};
struct encoded_field {
type::uLEB128 field_idx_diff;
access_flag access_flags;
};
struct encoded_method {
type::uLEB128 method_idx_diff;
type::uLEB128 access_flags;
type::uLEB128 code_off;
};
struct class_data_item {
type::uLEB128 static_fields_size;
type::uLEB128 instance_fields_size;
type::uLEB128 direct_methods_size;
type::uLEB128 virtual_methods_size;
encoded_field static_fields[static_fields_size];
encoded_field instance_fields[instance_fields_size];
encoded_method direct_methods[direct_methods_size];
encoded_method virtual_methods[virtual_methods_size];
};
struct class_def_item {
u32 class_idx;
u32 access_flags;
u32 superclass_idx;
u32 interfaces_off;
u32 source_file_idx;
u32 annotations_off;
u32 class_data_off;
//class_data_item *class_data_off:u32;
u32 static_values_off;
char class_name[] @ addressof(parent.type_ids[class_idx].type_name);
}[[name(class_name)]];
struct type_item {
u16 type_idx;
};
struct type_list {
u32 size;
type_item list[size];
};
struct code_item {
u16 registers_size;
u16 ins_size;
u16 outs_size;
u16 tries_size;
u32 debug_info_off;
u32 insns_size;
u16 insns[insns_size];
};
struct try_item {
u32 start_addr;
u16 insn_count;
u16 handler_off;
};
struct Dex {
header_item header;
string_id_item string_ids[header.string_ids_size] @ header.string_ids_off;
type_id_item type_ids[header.type_ids_size] @ header.type_ids_off;
proto_id_item proto_ids[header.proto_ids_size] @ header.proto_ids_off;
field_id_item field_ids[header.field_ids_size] @ header.field_ids_off;
method_id_item method_ids[header.method_ids_size] @ header.method_ids_off;
class_def_item class_defs[header.class_defs_size] @ header.class_defs_off;
u8 data[header.data_size] @header.data_off;
map_list map_list @ header.map_off;
u8 link_data[header.link_size] @ header.link_off;
};
Dex dex @ 0x00;

5594
patterns/dicom.hexpat Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,9 +1,11 @@
#pragma description Apple Disk Image Trailer (DMG)
#pragma endian big
#include <type/magic.pat>
#include <type/size.pat>
#include <type/guid.pat>
#include <std/mem.pat>
import type.magic;
import type.size;
import type.guid;
import std.mem;
// Parse DMG Structure per http://newosxbook.com/DMG.html
//

115
patterns/dsstore.hexpat Normal file
View File

@@ -0,0 +1,115 @@
#pragma description .DS_Store file format
// Apple macOS .DS_Store format
#pragma endian big
import std.io;
struct RecordEntry {
u32 length;
char16 filename[length];
char id[4];
// either blob or length
char type[4];
if (type == "blob") {
u32 blobCount;
u8 blobData[blobCount];
}
else if (type == "long") {
u32 value;
}
else if (type == "shor") {
u32 value;
}
else if (type == "bool") {
u8 value;
}
};
struct TreeBlock {
u32 mode;
u32 count;
RecordEntry entries[count];
};
struct BuddyRootBlockOffsets {
u32 count;
u8 reserved[4];
u32 addresses[count];
padding[(1024 - (count * 4))];
};
struct BuddyTableOfContentEntry {
u8 count;
char name[count];
u32 value;
};
struct BuddyTableOfContents {
u32 count;
BuddyTableOfContentEntry toc[count];
};
struct BuddyFreeList {
u32 count;
u32 offsets[count];
};
struct BuddyRootBlock {
BuddyRootBlockOffsets offsets;
BuddyTableOfContents toc;
BuddyFreeList freelists[32];
};
struct BuddyBlock {
u32 blockCount;
u8 reserved[4];
// padding for next multiple of 256 entries (1024 bytes)
u32 addresses[blockCount];
// u8 padding[paddingCount];
u32 directoryCount;
// directory entries
u8 count;
u8 name[count];
u32 blockNumber;
// free lists
// 32 free lists
BuddyRootBlockOffsets off[32];
};
struct BlocksList {
u32 blockId [[color("E57373")]];
u32 internalBlocks [[color("80CBC4")]];
u32 countRecords [[color("ffeb3b")]];
u32 countBlocks [[color("64b5f6")]];
u32 reserved [[color("a1887f")]];
};
struct BuddyAllocator {
char header[4];
u32 offsetBookkeeping;
u32 sizeBookkeeping;
u32 offsetBookkeeping2;
u32 offsetData;
u8 reserved[12];
BuddyRootBlock root @ offsetBookkeeping + 4;
std::print("TOC {} address 0x{:08x}",
root.toc.toc[0].value,
root.offsets.addresses[root.toc.toc[0].value] >> 0x5 << 0x5);
BlocksList blocks @ (root.offsets.addresses[root.toc.toc[0].value] >> 0x5 << 0x5) + 4;
std::print("Blocks start at address 0x{:08x}, size 0x{:04x}",
root.offsets.addresses[blocks.blockId] >> 0x5 << 0x5,
1 << (root.offsets.addresses[blocks.blockId] & 0x1f));
TreeBlock entries @ (root.offsets.addresses[blocks.blockId] >> 0x5 << 0x5) + 4;
};
BuddyAllocator buddy @0x04;

117
patterns/dted.hexpat Normal file
View File

@@ -0,0 +1,117 @@
#pragma description Digital Terrain Elevation Data
#pragma endian big
import std.core;
import std.io;
import std.mem;
import std.string;
enum Magic:u24 {
UHL = 0x55484C,
DSI = 0x445349,
ACC = 0x414343,
};
struct UHL {
Magic magic;
char one;
char lon[8];
char lat[8];
char lon_data_interval[4];
char lat_data_interval[4];
char accuracy[4];
char security_code[3];
char uniq_ref[12];
char lon_lines[4];
char lat_points[4];
char multi_accuracy;
char reserved[24];
};
struct DSI {
Magic magic;
char classification;
char stuff1[29];
char stuff2[26];
char product_level[5];
char uniq_ref[15];
char reserved[8];
char data_edition[2];
char match_version;
char maint_date[4];
char match_date[4];
char main_desc_code[4];
char producer_code[8];
char reserved2[16];
char product_spec[9];
char numbers[2];
char product_spec_date[4];
char vertical_datum[3];
char horizontal_datum[5];
char digitizing_system[10];
char compilation_date[4];
char reserved3[22];
char lat_origin[9];
char lon_origin[10];
char lat_sw_corner[7];
char lon_sw_corner[8];
char lat_nw_corner[7];
char lon_nw_corner[8];
char lat_ne_corner[7];
char lon_ne_corner[8];
char lat_se_corner[7];
char lon_se_corner[8];
char clockwise_orientation[9];
char lat_interval[4];
char lon_interval[4];
char lat_lines[4];
char lon_lines[4];
char partial_cell[2];
char reserved4[101];
char reserved5[100];
char reserved6[156];
};
struct ACCSub {
char abs_vertical_accuracy[4];
char abs_horizontal_accuracy[4];
char rel_vertical_accuracy[4];
char rel_horizontal_accuracy[4];
char num_coords[2];
char pairs[19*14];
};
struct ACC {
Magic magic;
char abs_horizontal_accuracy[4];
char abs_vertical_accuracy[4];
char rel_horizontal_accuracy[4];
char rel_vertical_accuracy[4];
char reserved1[4];
char reserved2[1];
char reserved3[31];
char multi_accuracy_outline[2]; // determines sub regions
ACCSub subs[9];
char reserved4[18];
char reserved5[69];
};
struct DataRecords {
char magic;
s24 data_block_count;
s16 lon_count;
s16 lat_count;
s16 elevation[std::string::parse_int(parent.dsi.lat_lines, 10)];
u32 checksum;
};
struct DTED {
UHL uhl;
DSI dsi;
ACC acc;
DataRecords records[std::string::parse_int(this.dsi.lon_lines, 10)];
};
DTED dted @ 0x00;

View File

@@ -1,11 +1,17 @@
#pragma author WerWolv
#pragma description ELF header in elf binaries
#pragma MIME application/x-executable
#pragma MIME application/x-elf
#pragma MIME application/x-coredump
#pragma MIME application/x-object
#pragma MIME application/x-sharedlib
#include <std/core.pat>
#include <std/mem.pat>
import std.core;
import std.io;
import std.mem;
using BitfieldOrder = std::core::BitfieldOrder;
using EI_ABIVERSION = u8;
using Elf32_Addr = u32;
@@ -457,56 +463,56 @@ enum VER_NEED : Elf32_Half {
};
bitfield SYMINFO_FLG {
padding : 10;
NOEXTDIRECT : 1;
DIRECTBIND : 1;
LAZYLOAD : 1;
COPY : 1;
RESERVED : 1;
DIRECT : 1;
} [[left_to_right]];
RESERVED : 1;
COPY : 1;
LAZYLOAD : 1;
DIRECTBIND : 1;
NOEXTDIRECT : 1;
padding : 10;
};
bitfield ST {
ST_BIND : 4;
ST_TYPE : 4;
} [[left_to_right]];
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 8)]];
bitfield SHF {
MASKPROC : 4;
MASKOS : 8;
UNKNOWN : 8;
COMPRESSED : 1;
TLS : 1;
GROUP : 1;
OS_NONCONFORMING : 1;
LINK_ORDER : 1;
INFO_LINK : 1;
STRINGS : 1;
MERGE : 1;
padding : 1;
EXECINSTR : 1;
ALLOC : 1;
WRITE : 1;
} [[left_to_right]];
ALLOC : 1;
EXECINSTR : 1;
padding : 1;
MERGE : 1;
STRINGS : 1;
INFO_LINK : 1;
LINK_ORDER : 1;
OS_NONCONFORMING : 1;
GROUP : 1;
TLS : 1;
COMPRESSED : 1;
UNKNOWN : 8;
MASKOS : 8;
MASKPROC : 4;
};
bitfield ELF32_R_INFO {
SYM : 8;
TYPE : 8;
} [[left_to_right]];
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 16)]];
bitfield ELF64_R_INFO {
SYM : 32;
TYPE : 32;
} [[left_to_right]];
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 64)]];
bitfield PF {
MASKPROC : 4;
MASKOS : 4;
padding : 17;
R : 1;
W : 1;
X : 1;
} [[left_to_right]];
W : 1;
R : 1;
padding : 17;
MASKOS : 4;
MASKPROC : 4;
};
struct E_IDENT {
char EI_MAG[4];
@@ -560,8 +566,8 @@ struct Elf32_Phdr {
PF p_flags;
Elf32_Word p_align;
if (p_offset > 0 && p_filesz > 0 && (p_offset + p_filesz) < std::mem::size() && p_filesz < std::mem::size())
u8 p_data[p_filesz] @ p_offset;
if (p_offset >= 0 && p_filesz > 0 && (p_offset + p_filesz) <= std::mem::size() && p_filesz <= std::mem::size())
u8 p_data[p_filesz] @ p_offset [[sealed]];
};
struct Elf64_Phdr {
@@ -574,8 +580,8 @@ struct Elf64_Phdr {
Elf64_Xword p_memsz;
Elf64_Xword p_align;
if (p_offset > 0 && p_filesz > 0 && (p_offset + p_filesz) < std::mem::size() && p_filesz < std::mem::size())
u8 p_data[p_filesz] @ p_offset;
if (p_offset >= 0 && p_filesz > 0 && (p_offset + p_filesz) <= std::mem::size() && p_filesz <= std::mem::size())
u8 p_data[p_filesz] @ p_offset [[sealed]];
};
struct Elf32_Chdr {
@@ -637,12 +643,12 @@ struct Elf32_Shdr {
} else if (sh_type == SHT::STRTAB) {
String stringTable[while($ < (sh_offset + sh_size))] @ sh_offset;
stringTableIndex = std::core::array_index();
} else if (sh_type == SHT::DYNSYM) {
} else if (sh_type == SHT::SYMTAB || sh_type == SHT::DYNSYM) {
Elf32_Sym symbolTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::INIT_ARRAY || sh_type == SHT::FINI_ARRAY) {
u32 pointer[while($ < (sh_offset + sh_size))] @ sh_offset;
} else {
u8 data[sh_size] @ sh_offset;
u8 data[sh_size] @ sh_offset [[sealed]];
}
}
} [[format("format_section_header")]];;
@@ -697,12 +703,12 @@ struct Elf64_Shdr {
} else if (sh_type == SHT::STRTAB) {
String stringTable[while($ < (sh_offset + sh_size))] @ sh_offset;
stringTableIndex = std::core::array_index();
} else if (sh_type == SHT::DYNSYM) {
} else if (sh_type == SHT::SYMTAB || sh_type == SHT::DYNSYM) {
Elf64_Sym symbolTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::INIT_ARRAY || sh_type == SHT::FINI_ARRAY) {
u32 pointer[while($ < (sh_offset + sh_size))] @ sh_offset;
} else {
u8 data[sh_size] @ sh_offset;
u8 data[sh_size] @ sh_offset [[sealed]];
}
}
} [[format("format_section_header")]];
@@ -711,14 +717,9 @@ fn format_section_header(auto shdr) {
u32 i = 0;
u32 nameAddress = addressof(elf.shdr[stringTableIndex].stringTable) + shdr.sh_name;
while (i < std::core::member_count(elf.shdr[stringTableIndex].stringTable)) {
if (nameAddress >= addressof(elf.shdr[stringTableIndex].stringTable[i]) && nameAddress < (addressof(elf.shdr[stringTableIndex].stringTable[i]) + sizeof(elf.shdr[stringTableIndex].stringTable[i])))
break;
i += 1;
}
String string @ nameAddress;
return elf.shdr[stringTableIndex].stringTable[i].value;
return string;
};
@@ -742,3 +743,78 @@ struct ELF {
};
ELF elf @ 0x00;
fn gen_shdr_disp_name(ref auto pattern, str member_name, u8 member_index) {
return std::format(
"{}@shdr[{}]({})",
member_name,
member_index,
format_section_header(pattern)
);
};
fn gen_shdr_sub_disp_name(ref auto pattern, str member_name, u8 member_index, u64 submember_index) {
return std::format(
"{}[{}]@shdr[{}]({})",
member_name,
submember_index,
member_index,
format_section_header(pattern)
);
};
fn gen_phdr_disp_name(ref auto pattern, str member_name, u8 member_index) {
return std::format(
"{}@phdr[{}]({})",
member_name,
member_index,
pattern.p_type
);
};
bool displaySectionNames in;
fn main() {
if (!displaySectionNames)
return;
for (u32 i = 0, i < std::core::member_count(elf.shdr), i += 1) {
if (std::core::has_member(elf.shdr[i], "stringTable")) {
std::core::set_display_name(
elf.shdr[i].stringTable,
gen_shdr_disp_name(elf.shdr[i], "stringTable", i)
);
for (u64 j = 0, j < std::core::member_count(elf.shdr[i].stringTable), j += 1) {
std::core::set_display_name(
elf.shdr[i].stringTable[j],
gen_shdr_sub_disp_name(elf.shdr[i], "String", i, j)
);
}
} else if (std::core::has_member(elf.shdr[i], "symbolTable")) {
std::core::set_display_name(
elf.shdr[i].symbolTable,
gen_shdr_disp_name(elf.shdr[i], "symbolTable", i)
);
} else if (std::core::has_member(elf.shdr[i], "pointer")) {
std::core::set_display_name(
elf.shdr[i].pointer,
gen_shdr_disp_name(elf.shdr[i], "pointer", i)
);
} else if (std::core::has_member(elf.shdr[i], "data")) {
std::core::set_display_name(
elf.shdr[i].data,
gen_shdr_disp_name(elf.shdr[i], "data", i)
);
}
}
for (u32 i = 0, i < std::core::member_count(elf.phdr), i += 1) {
if (std::core::has_member(elf.phdr[i], "p_data")) {
std::core::set_display_name(
elf.phdr[i].p_data,
gen_phdr_disp_name(elf.phdr[i], "p_data", i)
);
}
}
};

59
patterns/evtx.hexpat Normal file
View File

@@ -0,0 +1,59 @@
#pragma description MS Windows Vista Event Log
#pragma endian little
struct Header {
char signature[0x8];
u64 first_chunk_number;
u64 last_chunk_number;
u64 next_record_identifier;
u32 header_size;
u16 minor_format_version;
u16 major_format_version;
u16 header_block_size;
u16 number_of_chunks;
u8 unknown[0x4C];
u32 file_Flag;
u32 checkSum;
u8 unknown2[3968];
};
struct BinaryXML{
u8 fragment_header_token;
u8 major_version;
u8 minor_version;
u8 flags;
};
struct Event_Record{
u32 signature;
u32 size;
u64 event_record_identifier;
u64 written_data_amd_time;
BinaryXML binaryxml;
};
struct Chunk{
char signature[0x8];
u64 first_event_record_number;
u64 last_event_record_number;
u64 first_event_record_identifier;
u64 last_event_record_identifier;
u32 header_size;
u32 last_event_record_data_offset;
u32 free_space_offset;
u32 event_records_checksum;
u8 unknown[64];
u32 unknown2;
u32 checksum;
u8 common_string_offset_array[256];
u8 templatePtr[128];
Event_Record event_record;
};
struct Evtx {
Header header;
Chunk chunk;
};
Evtx evtx @ 0x00;

View File

@@ -0,0 +1,111 @@
#pragma author DmitriLeon2000
#pragma description Oska Software DeskMates FAS (Frames and Sequences) file
#pragma endian little
enum Compression : u32 {
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
};
struct Colors {
u8 blue;
u8 green;
u8 red;
u8 reserved;
};
struct FASHeader {
char name[50]; // name of the pack, may include garbage data
u16 version; // Format version number
s16 frameIDFirst; // assigned ID of the first frame
s16 frameIDLast; // assigned ID of the last frame
s32 width; // width of the animation (in pixels)
s32 height; // height of the animation (in pixels)
u16 frameSize; // size of the animation ((frameWidth * BPP + 31) // 32 * 4 * frameHeight)
u16 DblHeaderCount; // amount of BitmapInfoHeader pairs (one for color, one for masking)
};
struct BitmapInfoHeader { // bog-standard BitmapInfoHeaderV1
u32 biSize;
s32 biWidth;
s32 biHeight;
u16 biPlanes;
u16 biBitCount;
Compression compression;
u32 biSizeImage;
s32 biXPelsPerMeter;
s32 biYPelsPerMeter;
u32 biClrUsed;
u32 biClrImportant;
};
struct BitmapInfo {
BitmapInfoHeader header;
Colors colorMap[header.biClrUsed == 0 ?
1 << header.biBitCount : header.biClrUsed];
};
struct ExtraSprite {
char name[];
};
struct AnimSequence {
char name[];
char sequence[];
};
struct SeqHeader {
le u32 size;
le u32 count;
le u32 strPointers[count*2];
};
struct Bitmap {
u8 byte[fas.fasHeader.version >= 3 ? fas.fasHeader.frameSize + (fas.frameSizeHigh << 16) : fas.fasHeader.frameSize];
};
struct FramesHeader {
le u32 count;
le s16 frameID[count];
u8 frameDblHdrID[count];
};
struct FAS {
FASHeader fasHeader;
BitmapInfo dibHeaders[fasHeader.DblHeaderCount * 2];
if (fasHeader.version >= 1)
u8 charID;
if (fasHeader.version >= 2)
{
u32 extraEndOffset;
u16 extraSpritesCount;
if (extraSpritesCount)
// char extraSpriteList[touchOffset - 6];
ExtraSprite extraSprites[extraSpritesCount];
}
if (fasHeader.version >= 3)
le u16 frameSizeHigh;
le u16 touchColors;
if (touchColors)
{
u32 touchColorMap[touchColors];
u16 touchWidth;
u8 touchBitmap[this.touchWidth * fasHeader.height];
}
SeqHeader seqHeader;
if (fasHeader.version >= 1)
u8 filenameChecksum; // a checksum for a filename in ASCII
AnimSequence sequences[seqHeader.count];
FramesHeader framesHeader;
};
FAS fas @ 0x00;
Bitmap framesBitmap[fas.framesHeader.count] @ $;

View File

@@ -0,0 +1,87 @@
#pragma author DmitriLeon2000
#pragma description Oska Software DeskMates FAS (Frames and Sequences) file (Oska DeskMate versions 1.3 and 2.06)
#pragma endian little
enum Compression : u32 {
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
};
struct Colors {
u8 blue;
u8 green;
u8 red;
u8 reserved;
};
struct FASHeader {
s32 width; // width of the animation (in pixels)
s32 height; // height of the animation (in pixels)
u16 reserved1;
u16 reserved2;
u16 frameSizeCombined; // a sum of frameSizeColor and frameSizeMask
u16 frameSizeColor; // size of the animation ((frameWidth * 4 + 31) // 32 * 4 * frameHeight)
u16 frameSizeMask; // size of the animation's mask ((frameWidth * 1 + 31) // 32 * 4 * frameHeight)
u16 headerCount; // amount of DIB headers
};
struct BitmapInfoHeader { // bog-standard BitmapInfoHeaderV1
u32 biSize;
s32 biWidth;
s32 biHeight;
u16 biPlanes;
u16 biBitCount;
Compression compression;
u32 biSizeImage;
s32 biXPelsPerMeter;
s32 biYPelsPerMeter;
u32 biClrUsed;
u32 biClrImportant;
};
struct BitmapInfo {
BitmapInfoHeader header;
le u32 colorMap[header.biClrUsed == 0 ?
1 << header.biBitCount : header.biClrUsed];
};
struct AnimSequence {
char name[];
char sequence[];
};
struct SeqHeader {
le u32 size;
le u32 count;
le u32 strPointers[count*2];
};
struct Frame {
u8 colorBitmap[fas.fasHeader.frameSizeColor];
u8 maskBitmap[fas.fasHeader.frameSizeMask];
};
struct FramesHeader {
le u32 count;
le s16 frameID[count];
};
struct FAS {
FASHeader fasHeader;
BitmapInfo dibHeaders[fasHeader.headerCount];
SeqHeader seqHeader;
AnimSequence sequences[seqHeader.count];
FramesHeader framesHeader;
};
FAS fas @ 0x00;
Frame frames[fas.framesHeader.count] @ $;

125
patterns/fdt.hexpat Normal file
View File

@@ -0,0 +1,125 @@
#pragma author WerWolv
#pragma description Flat Linux Device Tree blob
#pragma endian big
import std.sys;
import std.io;
import std.core;
import type.magic;
import type.size;
// These are used in order for the children to be able to find strings
u64 fdt_addr;
u64 str_offset;
struct FDTHeader {
type::Magic<"\xD0\x0D\xFE\xED"> magic;
u32 totalsize;
u32 off_dt_struct;
u32 off_dt_strings;
u32 off_mem_rsvmap;
u32 version;
u32 last_comp_version;
u32 boot_cpuid_phys;
u32 size_dt_strings;
u32 size_dt_struct;
};
struct AlignTo<auto Alignment> {
padding[Alignment- ((($ - 1) % Alignment) + 1)];
} [[hidden]];
struct FDTReserveEntry {
u64 address;
type::Size<u64> size;
if (address == 0x00 && size == 0x00)
break;
};
enum FDTToken : u32 {
FDT_BEGIN_NODE = 0x00000001,
FDT_END_NODE = 0x00000002,
FDT_PROP = 0x00000003,
FDT_NOP = 0x00000004,
FDT_END = 0x00000009
};
union FDTPropValue<auto len> {
u32 prop_array[len / sizeof(u32)];
char string[len];
};
struct FDTProp {
FDTToken tok[[hidden]];
u32 len [[hidden]];
u32 nameoff [[hidden]];
if (len > 0) {
// if len is zero, value is absent and no need to include it
FDTPropValue<len> value;
}
AlignTo<4>;
char name[] @ fdt_addr + str_offset + nameoff;
std::core::set_display_name(this, name);
std::print(std::format("{:x} token {} len {} {}",
$, tok, len,nameoff));
FDTToken token = std::mem::read_unsigned($, 4, std::mem::Endian::Big);
if (token != FDTToken::FDT_PROP) {
break;
}
};
struct FDTNode {
FDTToken token = std::mem::read_unsigned($, 4, std::mem::Endian::Big);
if (token == FDTToken::FDT_BEGIN_NODE) {
FDTToken tok[[hidden]];
char name[];
AlignTo<4>;
std::core::set_display_name(this, name[0] ? name : "/");
token = std::mem::read_unsigned($, 4, std::mem::Endian::Big);
if(token == FDTToken::FDT_PROP) {
FDTProp props[while(true)];
}
token = std::mem::read_unsigned($, 4, std::mem::Endian::Big);
if(token == FDTToken::FDT_END_NODE) {
FDTToken[[hidden]];
break;
}
FDTNode children[while(true)][[inline]];
} else if(token == FDTToken::FDT_NOP) {
// TODO this may break the pattern, I've so far not encountered it
}
// ew duplication
token = std::mem::read_unsigned($, 4, std::mem::Endian::Big);
if(token == FDTToken::FDT_END_NODE) {
FDTToken[[hidden]];
break;
}
};
struct FDTStructureBlock {
FDTNode;
};
struct FDT {
FDTHeader header;
std::assert(header.version == 17, "Unsupported format version");
fdt_addr = addressof(this);
str_offset = header.off_dt_strings;
FDTStructureBlock structureBlocks @ fdt_addr + header.off_dt_struct;
FDTReserveEntry reserveEntries[while(true)] @ fdt_addr + header.off_mem_rsvmap;
};
std::mem::MagicSearch<"\xD0\x0D\xFE\xED", FDT> fdt @ std::mem::base_address();

View File

@@ -1,70 +0,0 @@
#pragma endian big
#include <std/sys.pat>
#include <std/io.pat>
#include <type/magic.pat>
#include <type/size.pat>
struct FDTHeader {
type::Magic<"\xD0\x0D\xFE\xED"> magic;
u32 totalsize;
u32 off_dt_struct;
u32 off_dt_strings;
u32 off_mem_rsvmap;
u32 version;
u32 last_comp_version;
u32 boot_cpuid_phys;
u32 size_dt_strings;
u32 size_dt_struct;
};
struct AlignTo<auto Alignment> {
padding[Alignment- ((($ - 1) % Alignment) + 1)];
};
struct FDTReserveEntry {
u64 address;
type::Size<u64> size;
if (address == 0x00 && size == 0x00)
break;
};
enum FDTToken : u32 {
FDT_BEGIN_NODE = 0x00000001,
FDT_END_NODE = 0x00000002,
FDT_PROP = 0x00000003,
FDT_NOP = 0x00000004,
FDT_END = 0x00000009
};
struct FDTStructureBlock {
FDTToken token;
if (token == FDTToken::FDT_BEGIN_NODE) {
char nodeName[];
AlignTo<4>;
} else if (token == FDTToken::FDT_END) {
break;
} else if (token == FDTToken::FDT_PROP) {
u32 len;
u32 nameoff;
char value[len];
AlignTo<4>;
char name[] @ parent.header.off_dt_strings + nameoff;
} else if (token == FDTToken::FDT_NOP || token == FDTToken::FDT_END_NODE) {
// Nothing to do
} else {
std::error(std::format("Invalid token at address 0x{:02X}", addressof(token)));
}
};
struct FDT {
FDTHeader header;
std::assert(header.version == 17, "Unsupported format version");
FDTStructureBlock structureBlocks[while(true)] @ header.off_dt_struct;
FDTReserveEntry reserveEntries[while(true)] @ header.off_mem_rsvmap;
};
FDT fdt @ 0x00;

1
patterns/ffx Submodule

Submodule patterns/ffx added at 199879e24a

View File

@@ -1,321 +1,328 @@
#include <std/sys.pat>
#include <std/core.pat>
#include <std/io.pat>
#pragma endian big
u32 sampleSize;
u32 bitsPerSample;
// METADATA
enum BLOCK_TYPE : u8 {
STREAMINFO = 0,
PADDING = 1,
APPLICATION = 2,
SEEKTABLE = 3,
VORBIS_COMMENT = 4,
CUESHEET = 5,
PICTURE = 6,
INVALID = 127
};
bitfield METADATA_BLOCK_HEADER {
lastMetadataBlock : 1;
blockType : 7;
length : 24;
} [[left_to_right]];
bitfield STREAMINFO_FLAGS {
sampleRate : 20;
numChannels : 3 [[format("format_channels")]];
bitsPerSample : 5;
numSamplesInStream : 36;
} [[inline, left_to_right]];
fn format_channels(u8 value) {
return value + 1;
};
struct METADATA_BLOCK_STREAMINFO {
u16 minBlockSize, maxBlockSize;
u24 minFrameSize, maxFrameSize;
STREAMINFO_FLAGS flags;
u128 md5Signature;
bitsPerSample = flags.bitsPerSample;
};
struct METADATA_BLOCK_PADDING {
padding[parent.header.length];
};
struct METADATA_BLOCK_APPLICATION {
char applicationID[4];
u8 applicationData[parent.header.length - sizeof(applicationID)];
};
struct SEEKPOINT {
u64 sampleNumber;
u64 byteOffset;
u16 numSamples;
};
struct METADATA_BLOCK_SEEKTABLE {
SEEKPOINT seekPoints[parent.header.length / 18];
};
struct VORBIS_USER_COMMENT {
le u32 length;
char comment[length];
};
struct METADATA_BLOCK_VORBIS_COMMENT {
le u32 vendorLength;
u8 vendor[vendorLength];
le u32 userCommentListLength;
VORBIS_USER_COMMENT userCommentList[userCommentListLength];
};
bitfield TRACK_FLAGS {
audioTrack : 1;
preEmphasis : 1;
padding : 6;
} [[inline]];
struct CUESHEET_TRACK_INDEX {
u64 sampleOffset;
u8 indexPointNumber;
padding[3];
};
struct CUESHEET_TRACK {
u64 trackOffset;
u8 trackNumber;
char trackISRC[12];
TRACK_FLAGS flags;
padding[13];
u8 numTrackIndexPoints;
CUESHEET_TRACK_INDEX indexes[numTrackIndexPoints];
};
struct METADATA_BLOCK_CUESHEET {
char mediaCatalogNumber[128];
u64 numLeadInSamples;
bool isCD;
padding[258];
u8 numTracks;
CUESHEET_TRACK tracks[numTracks];
};
enum PICTURE_TYPE : u32 {
Other = 0,
FileIcon = 1,
OtherFileIcon = 2,
CoverFront = 3,
CoverBack = 4,
LeafletPage = 5,
Media = 6,
LeadArtist = 7,
Artist = 8,
Conductor = 9,
Band = 10,
Composer = 11,
Lyricist = 12,
RecordingLocation = 13,
DuringRecording = 14,
DuringPerformance = 15,
MovieScreenCapture = 16,
ABrightColoredFish = 17,
Illustration = 18,
BandLogoType = 19,
PublisherLogoType = 20
};
struct METADATA_BLOCK_PICTURE {
PICTURE_TYPE pictureType;
u32 mimeTypeLength;
char mimeType[mineTypeLength];
u32 descriptionLength;
char description[descriptionLength];
u32 width, height;
u32 colorDepth;
u32 colorCount;
u32 pictureDataLength;
u8 pictureData[pictureDataLength];
};
// FRAME DATA
// TODO: THIS IS INCOMPLETE / NOT WORKING CURRENTLY
bitfield FRAME_HEADER_FLAGS {
syncCode : 14;
padding : 1;
blockingStrategy : 1;
blockSize : 4;
sampleRate : 4;
channelAssignment : 4;
sampleSize : 3;
padding : 1;
} [[inline]];
struct FRAME_HEADER {
FRAME_HEADER_FLAGS flags;
sampleSize = flags.sampleSize;
if (flags.blockingStrategy)
char16 sampleNumber[7];
else
char16 frameNumber[6];
if (flags.blockSize == 0b0110)
u8 blockSize;
else if (flags.blockSize == 0b0111)
u16 blockSize;
if (flags.sampleRate == 0b1100)
u8 sampleRate;
else if (flags.sampleRate == 0b1101 || flags.sampleRate == 0b1110)
u16 sampleRate;
u8 crc8;
};
struct FRAME_FOOTER {
u16 crc16;
};
bitfield SUBFRAME_HEADER {
padding : 1;
type : 6;
wastedBits : 1;
};
fn getBitsPerSample() {
if (sampleSize == 0b000)
return bitsPerSample;
else if (sampleSize == 0b001)
return 8;
else if (sampleSize == 0b010)
return 12;
else if (sampleSize == 0b100)
return 16;
else if (sampleSize == 0b101)
return 20;
else if (sampleSize == 0b110)
return 24;
};
bitfield SUBFRAME_CONSTANT {
value : getBitsPerSample();
};
bitfield SUBFRAME_VERBATIM {
value : getBitsPerSample() * parent.parent.header.flags.blockSize;
};
bitfield SUBFRAME_FIXED_VALUE {
value : getBitsPerSample() * (parent.parent.header.type & 0b111);
codingMethod : 2;
} [[inline]];
bitfield RESIDUAL_CODING_METHOD_PARTITIONED_RICE {
partitionOrder : 4;
riceParameter : 4;
if (riceParameter == 0b1111)
bitsPerSample : 5;
} [[right_to_left]];
bitfield RESIDUAL_CODING_METHOD_PARTITIONED_RICE2 {
partitionOrder : 4;
riceParameter : 5;
if (riceParameter == 0b11111)
bitsPerSample : 5;
} [[right_to_left]];
struct RESIDUAL {
if (parent.value.codingMethod == 0b00)
RESIDUAL_CODING_METHOD_PARTITIONED_RICE rice;
else if (parent.value.codingMethod == 0b01)
RESIDUAL_CODING_METHOD_PARTITIONED_RICE2 rice;
if ((parent.parent.header.type & 0b111) == 0b000)
u8 samples[(getBitsPerSample() * (parent.parent.parent.header.flags.blockSize - (parent.parent.header.type & 0b111))) / 8];
else if (std::core::array_index() != 0)
u8 samples[(getBitsPerSample() * (parent.parent.parent.header.flags.blockSize / (1 << rice.partitionOrder))) / 8];
else
u8 samples[(getBitsPerSample() * ((parent.parent.parent.header.flags.blockSize / (1 << rice.partitionOrder)) - (parent.parent.header.type & 0b111))) / 8];
};
struct SUBFRAME_FIXED {
SUBFRAME_FIXED_VALUE value;
RESIDUAL residual;
};
bitfield SUBFRAME_LPC_VALUE {
warmUpSamples : getBitsPerSample() * ((parent.header.type & 0b011111) + 1);
quantizedLinearPredictorCoefficient : 4;
quantizedLinearPredictorCoefficientShift : 5;
predictorCoefficients : quantizedLinearPredictorCoefficient * ((parent.header.type & 0b011111) + 1);
} [[inline]];
struct SUBFRAME_LPC {
SUBFRAME_LPC_VALUE value;
RESIDUAL residual;
};
struct SUBFRAME {
SUBFRAME_HEADER header;
if (header.type == 0b00000)
SUBFRAME_CONSTANT constant;
else if (header.type == 0b000001)
SUBFRAME_VERBATIM verbatim;
else if ((header.type >> 3) == 0b001 && (header.type & 0b111) <= 4)
SUBFRAME_FIXED fixed;
else if (header.type == 0b100000)
SUBFRAME_LPC lpc;
};
struct FRAME {
FRAME_HEADER header;
SUBFRAME subframes[parent.metadata[0].data.flags.numChannels + 1];
FRAME_FOOTER footer;
};
struct METADATA_BLOCK {
METADATA_BLOCK_HEADER header;
if (header.lastMetadataBlock)
break;
if (header.blockType == BLOCK_TYPE::STREAMINFO)
METADATA_BLOCK_STREAMINFO data;
else if (header.blockType == BLOCK_TYPE::PADDING)
METADATA_BLOCK_PADDING data;
else if (header.blockType == BLOCK_TYPE::APPLICATION)
METADATA_BLOCK_APPLICATION data;
else if (header.blockType == BLOCK_TYPE::VORBIS_COMMENT)
METADATA_BLOCK_VORBIS_COMMENT data;
else if (header.blockType == BLOCK_TYPE::CUESHEET)
METADATA_BLOCK_CUESHEET data;
else if (header.blockType == BLOCK_TYPE::PICTURE)
METADATA_BLOCK_PICTURE data;
else
std::error("Invalid metadata block type!");
};
struct STREAM {
char magic[4];
METADATA_BLOCK metadata[while(true)];
//FRAME frames[while(!std::mem::eof())];
};
STREAM stream @ 0x00;
#pragma author WerWolv
#pragma description Free Lossless Audio Codec, FLAC Audio Format
import std.sys;
import std.core;
import std.io;
#pragma endian big
using BitfieldOrder = std::core::BitfieldOrder;
u32 sampleSize;
u32 bitsPerSample;
// METADATA
enum BLOCK_TYPE : u8 {
STREAMINFO = 0,
PADDING = 1,
APPLICATION = 2,
SEEKTABLE = 3,
VORBIS_COMMENT = 4,
CUESHEET = 5,
PICTURE = 6,
INVALID = 127
};
bitfield METADATA_BLOCK_HEADER {
lastMetadataBlock : 1;
blockType : 7;
length : 24;
};
bitfield STREAMINFO_FLAGS {
sampleRate : 20;
numChannels : 3 [[format("format_channels")]];
bitsPerSample : 5;
numSamplesInStream : 36;
} [[inline]];
fn format_channels(u8 value) {
return value + 1;
};
struct METADATA_BLOCK_STREAMINFO {
u16 minBlockSize, maxBlockSize;
u24 minFrameSize, maxFrameSize;
STREAMINFO_FLAGS flags;
u128 md5Signature;
bitsPerSample = flags.bitsPerSample;
};
struct METADATA_BLOCK_PADDING {
padding[parent.header.length];
};
struct METADATA_BLOCK_APPLICATION {
char applicationID[4];
u8 applicationData[parent.header.length - sizeof(applicationID)];
};
struct SEEKPOINT {
u64 sampleNumber;
u64 byteOffset;
u16 numSamples;
};
struct METADATA_BLOCK_SEEKTABLE {
SEEKPOINT seekPoints[parent.header.length / 18];
};
struct VORBIS_USER_COMMENT {
le u32 length;
char comment[length];
};
struct METADATA_BLOCK_VORBIS_COMMENT {
le u32 vendorLength;
u8 vendor[vendorLength];
le u32 userCommentListLength;
VORBIS_USER_COMMENT userCommentList[userCommentListLength];
};
bitfield TRACK_FLAGS {
audioTrack : 1;
preEmphasis : 1;
} [[inline,bitfield_order(BitfieldOrder::LeastToMostSignificant, 8)]];
struct CUESHEET_TRACK_INDEX {
u64 sampleOffset;
u8 indexPointNumber;
padding[3];
};
struct CUESHEET_TRACK {
u64 trackOffset;
u8 trackNumber;
char trackISRC[12];
TRACK_FLAGS flags;
padding[13];
u8 numTrackIndexPoints;
CUESHEET_TRACK_INDEX indexes[numTrackIndexPoints];
};
struct METADATA_BLOCK_CUESHEET {
char mediaCatalogNumber[128];
u64 numLeadInSamples;
bool isCD;
padding[258];
u8 numTracks;
CUESHEET_TRACK tracks[numTracks];
};
enum PICTURE_TYPE : u32 {
Other = 0,
FileIcon = 1,
OtherFileIcon = 2,
CoverFront = 3,
CoverBack = 4,
LeafletPage = 5,
Media = 6,
LeadArtist = 7,
Artist = 8,
Conductor = 9,
Band = 10,
Composer = 11,
Lyricist = 12,
RecordingLocation = 13,
DuringRecording = 14,
DuringPerformance = 15,
MovieScreenCapture = 16,
ABrightColoredFish = 17,
Illustration = 18,
BandLogoType = 19,
PublisherLogoType = 20
};
struct METADATA_BLOCK_PICTURE {
PICTURE_TYPE pictureType;
u32 mimeTypeLength;
char mimeType[mimeTypeLength];
u32 descriptionLength;
char description[descriptionLength];
u32 width, height;
u32 colorDepth;
u32 colorCount;
u32 pictureDataLength;
u8 pictureData[pictureDataLength];
};
// FRAME DATA
// TODO: THIS IS INCOMPLETE / NOT WORKING CURRENTLY
bitfield FRAME_HEADER_FLAGS {
syncCode : 14;
padding : 1;
blockingStrategy : 1;
blockSize : 4;
sampleRate : 4;
channelAssignment : 4;
sampleSize : 3;
} [[inline,bitfield_order(BitfieldOrder::LeastToMostSignificant, 32)]];
struct FRAME_HEADER {
FRAME_HEADER_FLAGS flags;
sampleSize = flags.sampleSize;
if (flags.blockingStrategy)
char16 sampleNumber[7];
else
char16 frameNumber[6];
if (flags.blockSize == 0b0110)
u8 blockSize;
else if (flags.blockSize == 0b0111)
u16 blockSize;
if (flags.sampleRate == 0b1100)
u8 sampleRate;
else if (flags.sampleRate == 0b1101 || flags.sampleRate == 0b1110)
u16 sampleRate;
u8 crc8;
};
struct FRAME_FOOTER {
u16 crc16;
};
bitfield SUBFRAME_HEADER {
padding : 1;
type : 6;
wastedBits : 1;
};
fn getBitsPerSample() {
if (sampleSize == 0b000)
return bitsPerSample;
else if (sampleSize == 0b001)
return 8;
else if (sampleSize == 0b010)
return 12;
else if (sampleSize == 0b100)
return 16;
else if (sampleSize == 0b101)
return 20;
else if (sampleSize == 0b110)
return 24;
else
std::error(std::format("Invalid sample size {}.", sampleSize));
};
bitfield SUBFRAME_CONSTANT {
value : getBitsPerSample();
};
bitfield SUBFRAME_VERBATIM {
value : getBitsPerSample() * parent.parent.header.flags.blockSize;
};
bitfield SUBFRAME_FIXED_VALUE {
value : getBitsPerSample() * (parent.parent.header.type & 0b111);
codingMethod : 2;
} [[inline]];
bitfield RESIDUAL_CODING_METHOD_PARTITIONED_RICE {
partitionOrder : 4;
riceParameter : 4;
if (riceParameter == 0b1111)
bitsPerSample : 5;
};
bitfield RESIDUAL_CODING_METHOD_PARTITIONED_RICE2 {
partitionOrder : 4;
riceParameter : 5;
if (riceParameter == 0b11111)
bitsPerSample : 5;
};
struct RESIDUAL {
if (parent.value.codingMethod == 0b00)
RESIDUAL_CODING_METHOD_PARTITIONED_RICE rice;
else if (parent.value.codingMethod == 0b01)
RESIDUAL_CODING_METHOD_PARTITIONED_RICE2 rice;
if ((parent.parent.header.type & 0b111) == 0b000)
u8 samples[(getBitsPerSample() * (parent.parent.parent.header.flags.blockSize - (parent.parent.header.type & 0b111))) / 8];
else if (std::core::array_index() != 0)
u8 samples[(getBitsPerSample() * (parent.parent.parent.header.flags.blockSize / (1 << rice.partitionOrder))) / 8];
else
u8 samples[(getBitsPerSample() * ((parent.parent.parent.header.flags.blockSize / (1 << rice.partitionOrder)) - (parent.parent.header.type & 0b111))) / 8];
};
struct SUBFRAME_FIXED {
SUBFRAME_FIXED_VALUE value;
RESIDUAL residual;
};
bitfield SUBFRAME_LPC_VALUE {
warmUpSamples : getBitsPerSample() * ((parent.header.type & 0b011111) + 1);
quantizedLinearPredictorCoefficient : 4;
quantizedLinearPredictorCoefficientShift : 5;
predictorCoefficients : quantizedLinearPredictorCoefficient * ((parent.header.type & 0b011111) + 1);
} [[inline]];
struct SUBFRAME_LPC {
SUBFRAME_LPC_VALUE value;
RESIDUAL residual;
};
struct SUBFRAME {
SUBFRAME_HEADER header;
if (header.type == 0b00000)
SUBFRAME_CONSTANT constant;
else if (header.type == 0b000001)
SUBFRAME_VERBATIM verbatim;
else if ((header.type >> 3) == 0b001 && (header.type & 0b111) <= 4)
SUBFRAME_FIXED fixed;
else if (header.type == 0b100000)
SUBFRAME_LPC lpc;
};
struct FRAME {
FRAME_HEADER header;
SUBFRAME subframes[parent.metadata[0].data.flags.numChannels + 1];
FRAME_FOOTER footer;
};
struct METADATA_BLOCK {
METADATA_BLOCK_HEADER header;
if (header.lastMetadataBlock)
break;
if (header.blockType == BLOCK_TYPE::STREAMINFO)
METADATA_BLOCK_STREAMINFO data;
else if (header.blockType == BLOCK_TYPE::PADDING)
METADATA_BLOCK_PADDING data;
else if (header.blockType == BLOCK_TYPE::APPLICATION)
METADATA_BLOCK_APPLICATION data;
else if (header.blockType == BLOCK_TYPE::SEEKTABLE)
METADATA_BLOCK_SEEKTABLE data;
else if (header.blockType == BLOCK_TYPE::VORBIS_COMMENT)
METADATA_BLOCK_VORBIS_COMMENT data;
else if (header.blockType == BLOCK_TYPE::CUESHEET)
METADATA_BLOCK_CUESHEET data;
else if (header.blockType == BLOCK_TYPE::PICTURE)
METADATA_BLOCK_PICTURE data;
else
std::error("Invalid metadata block type!");
};
struct STREAM {
char magic[4];
METADATA_BLOCK metadata[while(true)];
//FRAME frames[while(!std::mem::eof())];
};
STREAM stream @ 0x00;

View File

@@ -1,168 +1,171 @@
#include <std/io.pat>
struct DiskTimeStamp {
u8 seconds, minutes, hours;
};
enum DiskProtection : u16 {
None = 0x0000,
CopyProtected = 0x5A5A
};
bitfield CHS {
h : 8;
s : 6;
c : 10;
} [[right_to_left, format("chs_formatter")]];
fn chs_formatter(CHS chs) {
return std::format("({:X}, {:X}, {:X}) | 0x{:X}", chs.c, chs.h, chs.s, (chs.c * 16 + chs.h) * 63 + (chs.s - 1));
};
enum PartitionStatus : u8 {
None = 0x00,
Active = 0x80
};
enum PartitionType : u8 {
EmptyPartitionEntry = 0x00,
FAT32_CHS = 0x0B,
FAT32_LBA = 0x0C
};
namespace fat32 {
u64 bytesPerCluster;
struct FSInfo {
u32 leadSignature;
padding[480];
u32 structSignature;
u32 freeClusterCount;
u32 nextFreeCluster;
padding[12];
u32 trailSignature;
};
bitfield SequenceNumber {
padding : 1;
lastLogical : 1;
padding : 1;
number : 5;
} [[left_to_right]];
enum EntryStatus : u8 {
Regular = 0x00,
DotEntry = 0x2E,
DeletedEntry = 0xE5
};
union EntryStatusOrSequenceNumber {
EntryStatus entryStatus;
SequenceNumber sequenceNumber;
};
bitfield Attributes {
readOnly : 1;
hidden : 1;
systemFile : 1;
volumeLabel : 1;
subdirectory : 1;
archive : 1;
padding : 2;
} [[right_to_left]];
struct DirEntry {
char fileName[8];
char extension[3];
Attributes attributes;
u8 reserved[10];
u16 time, date;
u16 startingCluster;
u32 fileSize;
u8 data[fileSize] @ startingCluster * bytesPerCluster;
};
struct VFATDirEntry {
EntryStatusOrSequenceNumber entryStatusOrSequenceNumber;
char16 name1[5];
Attributes attributes;
u8 type;
u8 nameChecksum;
char16 name2[6];
u16 startingCluster;
char16 name3[2];
if (entryStatusOrSequenceNumber.sequenceNumber.number > 1)
VFATDirEntry nextLogicalEntry;
else
DirEntry physicalEntry;
};
struct Partition {
u8 jmpCode[3];
char oemName[8];
u16 bytesPerSector;
u8 sectorsPerCluster;
u16 reservedAreaSize;
u8 numFats;
u16 rootEntryCount;
u16 numSectors;
u8 mediaType;
u16 fatSize;
u16 sectorsPerTrack;
u16 numHeads;
u32 numHiddenSectors;
u32 numFsSectors;
u32 numFatSectors;
u16 extFlags;
u16 fsVersion;
u32 rootCluster;
u16 fsInfoSector;
u16 backupBootSector;
padding[12];
u8 driveNumber;
padding[1];
u8 bootSignature;
u32 volumeID;
char volumeLabel[11];
char fsType[8];
u8 bootstrapCode[420];
u16 signature;
bytesPerCluster = (sectorsPerCluster * 1024) * bytesPerSector;
FSInfo fsInfo @ addressof(this) + fsInfoSector * bytesPerSector;
VFATDirEntry rootDirEntry @ addressof(this) + rootCluster * bytesPerCluster;
};
}
struct PartitionEntry {
PartitionStatus status;
CHS chsFirstSectorAddress;
PartitionType type;
CHS chsLastSectorAddress;
u32 lbaFirstSectorAddress;
u32 numSectors;
if (type == PartitionType::EmptyPartitionEntry)
continue;
else if (type == PartitionType::FAT32_CHS || type == PartitionType::FAT32_LBA)
fat32::Partition partition @ lbaFirstSectorAddress * 512;
};
struct MasterBootRecord {
u8 bootstrapCodeArea1[218];
padding[2];
u8 originalPhysicalDrive;
DiskTimeStamp diskTimeStamp;
u8 bootstrapCodeArea2[216];
u32 diskSignature;
DiskProtection diskProtection;
PartitionEntry partitionEntries[4];
u16 bootSignature;
};
#pragma author WerWolv
#pragma description Drive File System
import std.io;
struct DiskTimeStamp {
u8 seconds, minutes, hours;
};
enum DiskProtection : u16 {
None = 0x0000,
CopyProtected = 0x5A5A
};
bitfield CHS {
h : 8;
s : 6;
c : 10;
} [[format("chs_formatter")]];
fn chs_formatter(CHS chs) {
return std::format("({:X}, {:X}, {:X}) | 0x{:X}", chs.c, chs.h, chs.s, (chs.c * 16 + chs.h) * 63 + (chs.s - 1));
};
enum PartitionStatus : u8 {
None = 0x00,
Active = 0x80
};
enum PartitionType : u8 {
EmptyPartitionEntry = 0x00,
FAT32_CHS = 0x0B,
FAT32_LBA = 0x0C
};
namespace fat32 {
u64 bytesPerCluster;
struct FSInfo {
u32 leadSignature;
padding[480];
u32 structSignature;
u32 freeClusterCount;
u32 nextFreeCluster;
padding[12];
u32 trailSignature;
};
bitfield SequenceNumber {
padding : 1;
lastLogical : 1;
padding : 1;
number : 5;
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 8)]];
enum EntryStatus : u8 {
Regular = 0x00,
DotEntry = 0x2E,
DeletedEntry = 0xE5
};
union EntryStatusOrSequenceNumber {
EntryStatus entryStatus;
SequenceNumber sequenceNumber;
};
bitfield Attributes {
readOnly : 1;
hidden : 1;
systemFile : 1;
volumeLabel : 1;
subdirectory : 1;
archive : 1;
padding : 2;
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 8)]];
struct DirEntry {
char fileName[8];
char extension[3];
Attributes attributes;
u8 reserved[10];
u16 time, date;
u16 startingCluster;
u32 fileSize;
u8 data[fileSize] @ startingCluster * bytesPerCluster;
};
struct VFATDirEntry {
EntryStatusOrSequenceNumber entryStatusOrSequenceNumber;
char16 name1[5];
Attributes attributes;
u8 type;
u8 nameChecksum;
char16 name2[6];
u16 startingCluster;
char16 name3[2];
if (entryStatusOrSequenceNumber.sequenceNumber.number > 1)
VFATDirEntry nextLogicalEntry;
else
DirEntry physicalEntry;
};
struct Partition {
u8 jmpCode[3];
char oemName[8];
u16 bytesPerSector;
u8 sectorsPerCluster;
u16 reservedAreaSize;
u8 numFats;
u16 rootEntryCount;
u16 numSectors;
u8 mediaType;
u16 fatSize;
u16 sectorsPerTrack;
u16 numHeads;
u32 numHiddenSectors;
u32 numFsSectors;
u32 numFatSectors;
u16 extFlags;
u16 fsVersion;
u32 rootCluster;
u16 fsInfoSector;
u16 backupBootSector;
padding[12];
u8 driveNumber;
padding[1];
u8 bootSignature;
u32 volumeID;
char volumeLabel[11];
char fsType[8];
u8 bootstrapCode[420];
u16 signature;
bytesPerCluster = (sectorsPerCluster * 1024) * bytesPerSector;
FSInfo fsInfo @ addressof(this) + fsInfoSector * bytesPerSector;
VFATDirEntry rootDirEntry @ addressof(this) + rootCluster * bytesPerCluster;
};
}
struct PartitionEntry {
PartitionStatus status;
CHS chsFirstSectorAddress;
PartitionType type;
CHS chsLastSectorAddress;
u32 lbaFirstSectorAddress;
u32 numSectors;
if (type == PartitionType::EmptyPartitionEntry)
continue;
else if (type == PartitionType::FAT32_CHS || type == PartitionType::FAT32_LBA)
fat32::Partition partition @ lbaFirstSectorAddress * 512;
};
struct MasterBootRecord {
u8 bootstrapCodeArea1[218];
padding[2];
u8 originalPhysicalDrive;
DiskTimeStamp diskTimeStamp;
u8 bootstrapCodeArea2[216];
u32 diskSignature;
DiskProtection diskProtection;
PartitionEntry partitionEntries[4];
u16 bootSignature;
};
MasterBootRecord mbr @ 0x00;

361
patterns/gb.hexpat Normal file
View File

@@ -0,0 +1,361 @@
#pragma author WerWolv
#pragma description Gameboy ROM
#pragma MIME application/x-gameboy-rom
import type.size;
import std.string;
import std.mem;
bool uppercaseROMFeatures in;
bool brandedROMFeatures in;
bitfield CGB {
padding : 2;
PGB_MODE : 2;
padding : 2;
CGB_ONLY : 1 [[comment("gbcOnly")]];
CGB_SUPPORT : 1 [[comment("supportsGBC")]];
} [[comment("GBCFlags")]];
namespace format {
fn cart_size(u8 cartSizeType) {
u32 romSize = 32768 * (1 << cartSizeType);
u16 romBanks = 2 * (1 << cartSizeType);
if (cartSizeType == 0x52) {
romSize = 1153434;
romBanks = 72;
}
if (cartSizeType == 0x53) {
romSize = 1258291;
romBanks = 80;
}
return std::format("size: {}, banks: {}", type::impl::size_formatter(romSize), romBanks);
};
fn ram_type(u8 ramType) {
u16 ramSize;
u16 ramBanks;
if(ramType == 2) {
ramSize = 8192;
ramBanks = 1;
}
if(ramType == 3) {
ramSize = 32768;
ramBanks = 4;
}
if(ramType == 4) {
ramSize = 131072;
ramBanks = 16;
}
if(ramType == 5) {
ramSize = 65536;
ramBanks = 8;
}
return std::format("size: {}, banks: {}", type::impl::size_formatter(ramSize), ramBanks);
};
fn rom_features(u8 type) {
str result = "( ";
if (brandedROMFeatures) match(type) {
(0xFC): result += "pocket ";
(0xFD): result += "bandai ";
}
match(type) {
(0x00): result += "no_mbc";
(0x01): result += "mbc1";
(0x02): result += "mbc1 | ram";
(0x03): result += "mbc1 | ram | battery";
(0x05): result += "mbc2";
(0x06): result += "mbc2 | battery";
(0x08): result += "no_mbc | ram";
(0x09): result += "no_mbc | ram | battery";
(0x0B): result += "mmm01";
(0x0C): result += "mmm01 | ram";
(0x0D): result += "mmm01 | ram | battery";
(0x0F): result += "mbc3 | timer | battery";
(0x10): result += "mbc3 | timer | ram | battery";
(0x11): result += "mbc3";
(0x12): result += "mbc3 | ram";
(0x13): result += "mbc3 | ram | battery";
(0x19): result += "mbc5";
(0x1A): result += "mbc5 | ram";
(0x1B): result += "mbc5 | ram | battery";
(0x1C): result += "mbc5 | rumble";
(0x1D): result += "mbc5 | rumble | ram";
(0x1E): result += "mbc5 | rumble | ram | battery";
(0x20): result += "mbc6";
(0x22): result += "mbc7 | sensor | rumble";
(0xFC): result += "camera";
(0xFD): result += "tama5";
(0xFE): result += "huc3";
(0xFF): result += "huc1 | ram | battery";
}
if (uppercaseROMFeatures) result = std::string::to_upper(result);
return result + " )";
};
fn licensee_code(u8 code) {
match(code) {
(0x00): return "None";
(0x01 | 0x31): return "Nintendo";
(0x08 | 0x38): return "Capcom";
(0x09): return "Hot-B";
(0x0A | 0xE0): return "Jaleco";
(0x0B): return "Coconuts Japan";
(0x0C | 0x6E): return "Elite Systems";
(0x13 | 0x69): return "EA (Electronic Arts)";
(0x18): return "Hudsonsoft";
(0x19): return "ITC Entertainment";
(0x1A): return "Yanoman";
(0x1D): return "Japan Clary";
(0x1F | 0x4A | 0x61): return "Virgin Interactive";
(0x24): return "PCM Complete";
(0x25): return "San-X";
(0x28): return "Kotobuki Systems";
(0x29): return "Seta";
(0x30 | 0x70): return "Infogrames";
(0x32 | 0xA2 | 0xB2 | 0xC4): return "Bandai";
(0x33): return "See new licensee code";
(0x34 | 0xA4): return "Konami";
(0x35): return "HectorSoft";
(0x39 | 0x9D): return "Banpresto";
(0x3C): return ".Entertainment i";
(0x3E): return "Gremlin";
(0x41): return "Ubisoft";
(0x42 | 0xEB): return "Atlus";
(0x44 | 0x4D): return "Malibu";
(0x46 | 0xCF): return "Angel";
(0x47): return "Spectrum Holoby";
(0x49): return "Irem";
(0x50): return "Absolute";
(0x51 | 0xB0): return "Acclaim";
(0x52): return "Activision";
(0x53): return "American Sammy";
(0x54): return "GameTek";
(0x55): return "Park Place";
(0x56 | 0xDB | 0xFF): return "LJN";
(0x57): return "Matchbox";
(0x59): return "Milton Bradley";
(0x5A): return "Mindscape";
(0x5B): return "Romstar";
(0x5C | 0xD6): return "Naxat Soft";
(0x5D): return "Tradewest";
(0x60): return "Titus";
(0x67): return "Ocean Interactive";
(0x6F): return "Electro Brain";
(0x71): return "Interplay";
(0x72 | 0xAA): return "Broderbund";
(0x73): return "Sculptered Soft";
(0x75): return "The Sales Curve";
(0x78): return "t.hq";
(0x79): return "Accolade";
(0x7A): return "Triffix Entertainment";
(0x7C): return "Microprose";
(0x7F | 0xC2): return "Kemco";
(0x80): return "Misawa Entertainment";
(0x83): return "Lozc";
(0x86 | 0xC4): return "Tokuma Shoten Intermedia";
(0x8B): return "Bullet-Proof Software";
(0x8C): return "Vic Tokai";
(0x8E): return "Ape";
(0x8F): return "I'Max";
(0x91): return "Chunksoft Co.";
(0x92): return "Video System";
(0x93): return "Tsubaraya Productions Co.";
(0x95): return "Varie Corporation";
(0x96): return "Yonezawa/SPal";
(0x97): return "Kaneko";
(0x99): return "Arc";
(0x9A): return "Nihon Bussan";
(0x9B): return "Tecmo";
(0x9C): return "Imagineer";
(0x9F): return "Nova";
(0xA1): return "Hori Electric";
(0xA6): return "Kawada";
(0xA7): return "Takara";
(0xA9): return "Technos Japan";
(0xAC): return "Toei Animation";
(0xAD): return "Toho";
(0xAF): return "Namco";
(0xB1): return "ASCII or Nexsoft";
(0xB4): return "Square Enix";
(0xB6): return "HAL Laboratory";
(0xB7): return "SNK";
(0xB9 | 0xCE): return "Pony Canyon";
(0xBA): return "Culture Brain";
(0xBB): return "Sunsoft";
(0xBD): return "Sony Imagesoft";
(0xBF): return "Sammy";
(0xC0 | 0xD0): return "Taito";
(0xC3): return "Squaresoft";
(0xC5): return "Data East";
(0xC6): return "Tonkinhouse";
(0xC8): return "Koei";
(0xC9): return "UFL";
(0xCA): return "Ultra";
(0xCB): return "Vap";
(0xCC): return "Use Corporation";
(0xCD): return "Meldac";
(0xD1): return "Sofel";
(0xD2): return "Quest";
(0xD3): return "Sigma Enterprises";
(0xD4): return "ASK Kodansha Co.";
(0xD7): return "Copya System";
(0xDA): return "Tomy";
(0xDD): return "NCS";
(0xDE): return "Human";
(0xDF): return "Altron";
(0xE1): return "Towa Chiki";
(0xE2): return "Yutaka";
(0xE3): return "Varie";
(0xE5): return "Epoch";
(0xE7): return "Athena";
(0xE8): return "Asmik";
(0xE9): return "Natsume";
(0xEA): return "King Records";
(0xEE): return "IGS";
(0xF0): return "A Wave";
(0xF3): return "Extreme Entertainment";
}
return "Unknown Licensee";
};
fn new_licensee_code(str a) {
if (std::mem::read_unsigned(0x14B, 1) != 0x33) return "See old licensee code";
match(a) {
("00"): return "None";
("01"): return "Nintendo R&D1";
("08"): return "Capcom";
("13"): return "Electronic Arts";
("18"): return "Hudson Soft";
("19"): return "b-ai";
("20"): return "KSS";
("22"): return "pow";
("24"): return "PCM Complete";
("25"): return "san-x";
("28"): return "Kemco Japan";
("29"): return "seta";
("30"): return "Viacom";
("31"): return "Nintendo";
("32"): return "Bandai";
("33"): return "Ocean/Acclaim";
("34"): return "Konami";
("35"): return "Hector";
("37"): return "Taito";
("38"): return "Hudson";
("39"): return "Banpresto";
("41"): return "Ubi Soft";
("42"): return "Atlus";
("44"): return "Malibu";
("46"): return "Angel";
("47"): return "Bullet-Proof Software";
("49"): return "irem";
("50"): return "Absolute";
("51"): return "Acclaim";
("52"): return "Activision";
("53"): return "American Sammy";
("54"): return "Konami";
("55"): return "Hi tech entertainment";
("56"): return "LJN";
("57"): return "Matchbox";
("58"): return "Mattel";
("59"): return "Milton Bradley";
("60"): return "Titus";
("61"): return "Virgin";
("64"): return "LucasArts";
("67"): return "Ocean";
("69"): return "Electronic Arts";
("70"): return "Infogrames";
("71"): return "Interplay";
("72"): return "Broderbund";
("73"): return "sculptured";
("75"): return "sci";
("78"): return "THQ";
("79"): return "Accolade";
("80"): return "misawa";
("83"): return "lozc";
("86"): return "Tokuma Shoten Intermedia";
("87"): return "Tsukuda Original";
("91"): return "Chunksoft";
("92"): return "Video system";
("93"): return "Ocean/Acclaim";
("95"): return "Varie";
("96"): return "Yonezawa/spal";
("97"): return "Kaneko";
("99"): return "Pack in soft";
("A4"): return "Konami (Yu-Gi-Oh!)";
}
return "Unknown";
};
using CGB;
fn null_cgb_flags(CGB flags) {
return "NO_CGB";
};
}
enum DestinationType : u8 {
Japanese,
NonJapanese
};
enum SGBFlagType : u8 {
NoSGBFunctionality,
SGBFunctionality = 0x03
};
struct Header {
char title[16-(std::mem::read_unsigned(addressof(this)+15, 1) >= 0x80)];
if (std::mem::read_unsigned(addressof(this)+15, 1) == 0)
CGB gbcFlags @ $-1 [[format("format::null_cgb_flags")]];
else if (std::mem::read_unsigned(addressof(this)+15, 1) >= 0x80)
CGB gbcFlags;
char newLicenseeCode[2] [[format("format::new_licensee_code")]];
SGBFlagType sgbFlag;
u8 romType [[format("format::rom_features")]];
u8 cartSize [[format("format::cart_size")]];
u8 ramType [[format("format::ram_type")]];
DestinationType destination;
u8 licenseeCode [[format("format::licensee_code")]];
u8 maskROMVersionNumber;
u8 headerChecksum [[comment("Checksum for: $134-$14C")]];
u16 globalChecksum [[comment("Checksum for entire range")]];
};
bitfield pixel_packed {
p1 : 1;
p2 : 1;
p3 : 1;
p4 : 1;
p5 : 1;
p6 : 1;
p7 : 1;
p8 : 1;
};
struct CartridgeStart {
u8 entryCode[4];
pixel_packed logo[0x30];
Header header;
};
struct JumpVectors {
u8 rst1[8] [[comment("rst $1")]];
u8 rst2[8] [[comment("rst $2")]];
u8 rst3[8] [[comment("rst $3")]];
u8 rst4[8] [[comment("rst $4")]];
u8 rst5[8] [[comment("rst $5")]];
u8 rst6[8] [[comment("rst $6")]];
u8 rst7[8] [[comment("rst $7")]];
u8 rst8[8] [[comment("rst $8")]];
u8 int1[8] [[comment("vblank")]];
u8 int2[8] [[comment("lcd stat")]];
u8 int3[8] [[comment("timer")]];
u8 int4[8] [[comment("serial")]];
u8 int5[8] [[comment("joypad")]];
};
JumpVectors jumpVectors @ 0x00 [[comment("Instructions called on interrupts or RST instructions")]];
CartridgeStart cartridgeStart @ 0x100;

215
patterns/gguf.hexpat Normal file
View File

@@ -0,0 +1,215 @@
// https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
// https://github.com/openxla/iree/blob/main/runtime/src/iree/io/formats/gguf/gguf_parser.c
#pragma description GGUF v3 File Format Patter
#pragma authors @leonjza, jessie @ imhex discord
#pragma pattern_limit 300000
enum ggml_type: u32 {
GGML_TYPE_F32 = 0,
GGML_TYPE_F16 = 1,
GGML_TYPE_Q4_0 = 2,
GGML_TYPE_Q4_1 = 3,
// GGML_TYPE_Q4_2 = 4, support has been removed
// GGML_TYPE_Q4_3 = 5, support has been removed
GGML_TYPE_Q5_0 = 6,
GGML_TYPE_Q5_1 = 7,
GGML_TYPE_Q8_0 = 8,
GGML_TYPE_Q8_1 = 9,
GGML_TYPE_Q2_K = 10,
GGML_TYPE_Q3_K = 11,
GGML_TYPE_Q4_K = 12,
GGML_TYPE_Q5_K = 13,
GGML_TYPE_Q6_K = 14,
GGML_TYPE_Q8_K = 15,
GGML_TYPE_IQ2_XXS = 16,
GGML_TYPE_IQ2_XS = 17,
GGML_TYPE_IQ3_XXS = 18,
GGML_TYPE_IQ1_S = 19,
GGML_TYPE_IQ4_NL = 20,
GGML_TYPE_IQ3_S = 21,
GGML_TYPE_IQ2_S = 22,
GGML_TYPE_IQ4_XS = 23,
GGML_TYPE_I8 = 24,
GGML_TYPE_I16 = 25,
GGML_TYPE_I32 = 26,
GGML_TYPE_I64 = 27,
GGML_TYPE_F64 = 28,
GGML_TYPE_IQ1_M = 29,
GGML_TYPE_COUNT,
};
enum gguf_metadata_value_type: u32 {
// The value is a 8-bit unsigned integer.
GGUF_METADATA_VALUE_TYPE_UINT8 = 0,
// The value is a 8-bit signed integer.
GGUF_METADATA_VALUE_TYPE_INT8 = 1,
// The value is a 16-bit unsigned little-endian integer.
GGUF_METADATA_VALUE_TYPE_UINT16 = 2,
// The value is a 16-bit signed little-endian integer.
GGUF_METADATA_VALUE_TYPE_INT16 = 3,
// The value is a 32-bit unsigned little-endian integer.
GGUF_METADATA_VALUE_TYPE_UINT32 = 4,
// The value is a 32-bit signed little-endian integer.
GGUF_METADATA_VALUE_TYPE_INT32 = 5,
// The value is a 32-bit IEEE754 floating point number.
GGUF_METADATA_VALUE_TYPE_FLOAT32 = 6,
// The value is a boolean.
// 1-byte value where 0 is false and 1 is true.
// Anything else is invalid, and should be treated as either the model being invalid or the reader being buggy.
GGUF_METADATA_VALUE_TYPE_BOOL = 7,
// The value is a UTF-8 non-null-terminated string, with length prepended.
GGUF_METADATA_VALUE_TYPE_STRING = 8,
// The value is an array of other values, with the length and type prepended.
///
// Arrays can be nested, and the length of the array is the number of elements in the array, not the number of bytes.
GGUF_METADATA_VALUE_TYPE_ARRAY = 9,
// The value is a 64-bit unsigned little-endian integer.
GGUF_METADATA_VALUE_TYPE_UINT64 = 10,
// The value is a 64-bit signed little-endian integer.
GGUF_METADATA_VALUE_TYPE_INT64 = 11,
// The value is a 64-bit IEEE754 floating point number.
GGUF_METADATA_VALUE_TYPE_FLOAT64 = 12,
};
// A string in GGUF.
struct gguf_string_t {
// The length of the string, in bytes.
u64 len;
// The string as a UTF-8 non-null-terminated string.
char string[len];
};
struct gguf_metadata_value_t {
gguf_metadata_value_type type;
u64 length;
match(type) {
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT8): u8 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT8): s8 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT16): u16 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT16): s16 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT32): u32 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT32): s32 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT32): float value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_BOOL): bool value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_STRING): gguf_string_t value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT64): u64 value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT64): double value[length];
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_ARRAY): double value[length];
}
};
struct gguf_metadata_value {
gguf_metadata_value_type type;
match(type) {
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT8): u8 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT8): s8 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT16): u16 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT16): s16 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT32): u32 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_INT32): s32 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT32): float value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_BOOL): bool value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_STRING): gguf_string_t value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_UINT64): u64 value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_FLOAT64): double value;
(gguf_metadata_value_type::GGUF_METADATA_VALUE_TYPE_ARRAY): gguf_metadata_value_t value;
}
};
struct gguf_metadata_kv_t {
// The key of the metadata. It is a standard GGUF string, with the following caveats:
// - It must be a valid ASCII string.
// - It must be a hierarchical key, where each segment is `lower_snake_case` and separated by a `.`.
// - It must be at most 2^16-1/65535 bytes long.
// Any keys that do not follow these rules are invalid.
gguf_string_t key;
// The type of the value.
// Must be one of the `gguf_metadata_value_type` values.
// gguf_metadata_value_type value_type;
// The value.
gguf_metadata_value value;
};
struct gguf_header_t {
// Magic number to announce that this is a GGUF file.
// Must be `GGUF` at the byte level: `0x47` `0x47` `0x55` `0x46`.
// Your executor might do little-endian byte order, so it might be
// check for 0x46554747 and letting the endianness cancel out.
// Consider being *very* explicit about the byte order here.
u32 magic;
// The version of the format implemented.
// Must be `3` for version described in this spec, which introduces big-endian support.
//
// This version should only be increased for structural changes to the format.
// Changes that do not affect the structure of the file should instead update the metadata
// to signify the change.
u32 version;
// The number of tensors in the file.
// This is explicit, instead of being included in the metadata, to ensure it is always present
// for loading the tensors.
u64 tensor_count;
// The number of metadata key-value pairs.
u64 metadata_kv_count;
// The metadata key-value pairs.
gguf_metadata_kv_t metadata_kv[metadata_kv_count];
};
struct gguf_tensor_info_t {
// The name of the tensor. It is a standard GGUF string, with the caveat that
// it must be at most 64 bytes long.
gguf_string_t name;
// The number of dimensions in the tensor.
// Currently at most 4, but this may change in the future.
u32 n_dimensions;
// The dimensions of the tensor.
u64 dimensions[n_dimensions];
// The type of the tensor.
ggml_type type;
// The offset of the tensor's data in this file in bytes.
//
// This offset is relative to `tensor_data`, not to the start
// of the file, to make it easier for writers to write the file.
// Readers should consider exposing this offset relative to the
// file to make it easier to read the data.
//
// Must be a multiple of `ALIGNMENT`. That is, `align_offset(offset) == offset`.
u64 offset;
};
struct gguf_file_t {
// The header of the file.
gguf_header_t header;
// Tensor infos, which can be used to locate the tensor data.
gguf_tensor_info_t tensor_infos[header.tensor_count];
// Padding to the nearest multiple of `ALIGNMENT`.
//
// That is, if `sizeof(header) + sizeof(tensor_infos)` is not a multiple of `ALIGNMENT`,
// this padding is added to make it so.
//
// This can be calculated as `align_offset(position) - position`, where `position` is
// the position of the end of `tensor_infos` (i.e. `sizeof(header) + sizeof(tensor_infos)`).
u8 _padding[];
// Tensor data.
//
// This is arbitrary binary data corresponding to the weights of the model. This data should be close
// or identical to the data in the original model file, but may be different due to quantization or
// other optimizations for inference. Any such deviations should be recorded in the metadata or as
// part of the architecture definition.
//
// Each tensor's data must be stored within this array, and located through its `tensor_infos` entry.
// The offset of each tensor's data must be a multiple of `ALIGNMENT`, and the space between tensors
// should be padded to `ALIGNMENT` bytes.
u8 tensor_data[];
};
gguf_file_t GGUF @ 0x00;

View File

@@ -1,153 +1,185 @@
#include <std/mem.pat>
#pragma description GIF image files
#pragma MIME image/gif
#pragma bitfield_order left_to_right
// Extension Labels
#define LABEL_GC 0xF9
#define LABEL_COMMENT 0xFE
#define LABEL_APPLICATION 0xFF
#define LABEL_PLAINTEXT 0x01
// gif89a
// Indentifier Magics
#define IMAGE_SEPERATOR_MAGIC 0x2C
#define EXTENSION_INTRODUCER_MAGIC 0x21
#define GIF_TRAILER_MAGIC 0x3B
struct data_subblock_t {
u8 block_size;
u8 data_values[block_size];
import std.io;
import std.core;
import std.mem;
import std.string;
import std.math;
import type.magic;
bitfield GCT_Flags {
size : 3 [[comment("physical size = 2^(flags.size + 1)")]];
sort : 1 [[comment("Indicates if the table is sorted by importance")]];
colorRes : 3 [[comment("Indicates the richness of the original pallet")]];
enabled : 1;
};
bitfield ImageDescriptorFlags {
lctSize: 3;
reserved: 2;
lctSort: 1;
interlaceFlag: 1;
lctEnable: 1;
};
bitfield GCE_Flags {
transparent : 1;
userInput : 1;
disposalMode : 3 [[format("format::dispose_enum")]];
reserved : 3;
};
struct data_subblocks_t {
data_subblock_t data[while (std::mem::read_unsigned($, 1) != 0x00)];
u8 block_terminator; // 0x00
} [[inline]];
struct Color {
u8 red;
u8 green;
u8 blue;
} [[color(std::format("{:02X}{:02X}{:02X}", red, green, blue))]];
struct header_t {
char header[3];
struct _SubBlocks {
u8 data_size;
u8 data[data_size];
};
struct DataSubBlocks {
_SubBlocks subBlocks[while(std::mem::read_unsigned($, 1) != 0)];
};
struct Header {
type::Magic<"GIF"> magic;
char version[3];
};
bitfield lsd_fields_t {
global_color_table_flag : 1;
color_resolution : 3;
sort_flag : 1;
size_of_global_color_table : 3;
} [[inline]];
struct logical_screen_descriptor_t {
struct LogicalScreenDescriptor {
u16 width;
u16 height;
lsd_fields_t fields;
u8 background_color_index;
u8 pixel_aspect_ratio;
GCT_Flags gctFlags;
u8 bgColorIndex;
u8 pixelAscpet;
if (gctFlags.enabled) {
Color gct[std::math::pow(2, gctFlags.size + 1)];
}
};
struct color_table_entry_t {
u8 red;
u8 green;
u8 blue;
struct ImageDescriptor {
u16 imageLeftPosition;
u16 imageTopPosition;
u16 imageWidth;
u16 imageHeight;
ImageDescriptorFlags flags;
if(flags.lctEnable) {
Color lct[std::math::pow(2, flags.lctSize + 1)];
}
u8 lzwMinCode [[comment("This byte determines the initial number of bits used for LZW codes in the image data")]];
DataSubBlocks lzwCompressedData [[comment("Data is pallet indecies either into current LDC or GCT and is compressed using LZW")]];
};
bitfield gce_fields_t {
padding : 3;
disposal_method : 3;
user_input_flag : 1;
transparent_color_flag : 1;
} [[inline]];
struct graphic_coltrol_extension_t {
u8 extension_introducer; // 0x21
u8 graphic_control_label; // 0xf9
u8 block_size;
gce_fields_t fields;
u16 delay_time;
u8 transparent_color_index;
u8 block_terminator;
struct CommentExtension {
DataSubBlocks commentData;
};
struct comment_extension_t {
u8 extension_introducer; // 0x21
u8 comment_label; // 0xfe
data_subblocks_t comment_data;
struct ApplicationExtension {
u8 blockSize;
char identifier[8];
char authenticationCode[3];
DataSubBlocks applicationData;
};
struct plaintext_extension_t {
u8 extension_introducer; // 0x21
u8 plain_text_label; // 0x01
u8 block_size; // 12
u16 text_grid_left_position;
u16 text_grid_top_position;
u16 text_grid_width;
u16 text_grid_height;
u8 character_cell_width;
u8 character_cell_height;
u8 text_foreground_color_index;
u8 text_background_color_index;
data_subblocks_t plain_text_data;
};
struct application_extension_t {
u8 extension_introducer; // 0x21
u8 extension_label; // 0xff
u8 block_size; // 11
u8 application_identifier[8];
u8 application_authentication_code[3];
struct PlainTextExtension {
u8 blockSize;
u16 textGridLeftPos;
u16 textGridTopPos;
u16 textGridWidth;
u16 textGridHeight;
u8 charCellWidth;
u8 charCellHeight;
u8 textForegroundColorIndex;
u8 textBackgroundColorIndex;
DataSubBlocks plainTextData;
data_subblocks_t data;
};
bitfield id_fields_t {
local_color_table : 1;
interlace_flag : 1;
sort_fla : 1;
padding : 2;
size_of_local_color_table : 3;
} [[inline]];
struct image_descriptor_t {
u8 image_separator; // 0x2c
u16 image_left_position;
u16 image_top_position;
u16 image_width;
u16 image_height;
id_fields_t fields;
struct GraphicControlExtension {
u8 blockSize;
std::assert_warn(blockSize == 4, "Unexpected GCE block size");
GCE_Flags flags;
u16 delay [[format("format::delay")]];
u8 transparentColorIndex;
};
struct table_based_image_t {
u8 lzw_minimum_code_size;
data_subblocks_t data;
struct Extension {
u8 label [[format("format::extension_name")]];
if(label == LABEL_GC) GraphicControlExtension gce [[inline]];
if(label == LABEL_COMMENT) CommentExtension c [[inline]];
if(label == LABEL_APPLICATION) ApplicationExtension a [[inline]];
if(label == LABEL_PLAINTEXT) PlainTextExtension p [[inline]];
};
struct gif_block_t {
if (std::mem::read_unsigned($, 1) == 0x21) {
if (std::mem::read_unsigned($+1, 1) == 0x01) {
plaintext_extension_t plaintext_extension;
}
else if (std::mem::read_unsigned($+1, 1) == 0xf9) {
graphic_coltrol_extension_t graphic_coltrol_extension;
}
else if (std::mem::read_unsigned($+1, 1) == 0xfe) {
comment_extension_t comment;
} else {
application_extension_t application_extension;
}
}
else if (std::mem::read_unsigned($, 1) == 0x2c) {
image_descriptor_t image_descriptor;
if (image_descriptor.fields.local_color_table) {
color_table_entry_t local_color_table[1<<(image_descriptor.fields.size_of_local_color_table + 1)];
}
table_based_image_t image;
}
} [[inline]];
struct Block {
u8 identifier [[format("format::identifier_name")]];
if(identifier == IMAGE_SEPERATOR_MAGIC) ImageDescriptor i [[inline]];
if(identifier == EXTENSION_INTRODUCER_MAGIC) Extension e [[inline]];
if(identifier == GIF_TRAILER_MAGIC) break;
u8 blockTerminator;
} [[format("format::block_name")]];
struct gif_t {
header_t header;
logical_screen_descriptor_t logical_screen_descriptor;
if (logical_screen_descriptor.fields.global_color_table_flag) {
color_table_entry_t global_color_table[1<<(logical_screen_descriptor.fields.size_of_global_color_table + 1)];
}
namespace format {
fn dispose_enum(u8 value) {
if(value == 0x00) return "Do nothing";
if(value == 0x01) return "Do not remove pixels";
if(value == 0x02) return "Restore background pixels";
if(value == 0x03) return "Restore previous pixels";
};
gif_block_t blocks[while (std::mem::read_unsigned($, 1) != 0x3b)];
fn extension_name(u8 label) {
if(label == LABEL_GC) return "Graphical Control Extension";
if(label == LABEL_COMMENT) return "Comment Extension";
if(label == LABEL_APPLICATION) return "Application Extension";
if(label == LABEL_PLAINTEXT) return "Plaintext Extension";
return "Unknown Extension";
};
u8 trailer; // 0x3b
};
fn block_name(ref Block b) {
if(b.identifier == IMAGE_SEPERATOR_MAGIC) return "Image Descriptor";
if(b.identifier == EXTENSION_INTRODUCER_MAGIC) return "Extension";
if(b.identifier == GIF_TRAILER_MAGIC) return "Trailer";
return "Unknown Block Type";
};
gif_t gif @ 0x00;
fn identifier_name(u8 identifier) {
if(identifier == IMAGE_SEPERATOR_MAGIC) return "Image Separator";
if(identifier == EXTENSION_INTRODUCER_MAGIC) return "Extension Introducer";
if(identifier == GIF_TRAILER_MAGIC) return "GIF Trailer";
return "Unknown Identifier";
};
fn delay(u16 delay) {
if(delay == -1) return "forever";
else if(delay < 2) return "100ms";
else return std::string::to_string(delay * 10) + "ms";
return "notime";
};
}
struct Gif {
u8 data[std::mem::size()] [[no_unique_address, hidden]];
Header header;
std::assert_warn(header.version == "89a" || header.version == "87a", "Unsupported format version");
LogicalScreenDescriptor logicalScreenDescriptor;
Block blocks[while(!std::mem::eof())];
} [[hex::visualize("image", this.data)]];
Gif gif @ 0x00;

68
patterns/gltf.hexpat Normal file
View File

@@ -0,0 +1,68 @@
/**
* @file <gltf.pat> ImHex Pattern for glTF binary files.
*
* Copyright (c) 2023 H. Utku Maden <utkumaden@hotmail.com>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#pragma description TODO
import std.mem;
import std.io;
/**
* @brief The glTF magic section.
*/
struct gltf_magic_t {
char magic[4]; /**< The magic value. Must be "glTF" */
u32 version; /**< The version. Must be 2 for glTF 2.0. */
u32 length; /**< Length of the file in bytes, including magic section. */
};
/**
* @brief Enumeration of well defined glTF chunks.
*/
enum gltf_chunk_type_t : u32 {
JSON = 1313821514 /* "JSON" */, /**< JSON chunk. */
BIN = 5130562 /* "BIN\0" */, /**< Binary data chunk. Could be an image or model data. */
};
/**
* @brief A glTF chunk.
*/
struct gltf_chunk_t {
u32 length; /**< Length of this chunk. */
gltf_chunk_type_t type [[format("gltf_format")]]; /**< Type of the chunk. JSON or BIN expected. */
u8 string[length]; /**< The chunk data. */
};
fn gltf_format(gltf_chunk_type_t x)
{
if (x == gltf_chunk_type_t::JSON) return "JSON";
else if (x == gltf_chunk_type_t::BIN) return "BIN";
return "";
};
gltf_magic_t magic @ 0x00;
gltf_chunk_t chunks[while(!std::mem::eof())] @ $;
if (magic.magic != "glTF")
std::error("This file might not be a glTF file, expected \"glTF\", got %s", magic.magic);

View File

@@ -1,77 +1,82 @@
#pragma MIME application/gzip
#include <type/time.pat>
#include <type/size.pat>
#include <std/mem.pat>
bitfield Flags {
FTEXT : 1;
FHCRC : 1;
FEXTRA : 1;
FNAME : 1;
FCOMMENT : 1;
padding : 3;
} [[right_to_left]];
bitfield ExtraFlags {
padding : 1;
maximumCompression : 1;
fastestCompression : 1;
padding : 5;
};
enum CompressionMethod : u8 {
Reserved = 0 ... 7,
Deflate = 8
};
enum OperatingSystemID : u8 {
FATFileSystem = 0x00,
Amiga = 0x01,
VMS = 0x02,
Unix = 0x03,
VM_CMS = 0x04,
AtariTOS = 0x05,
HPFSFileSystem = 0x06,
Macintosh = 0x07,
ZSystem = 0x08,
CP_M = 0x09,
TOPS_20 = 0x0A,
NTFSFileSystem = 0x0B,
QDOS = 0x0C,
AcordRISCOS = 0x0D,
Unknown = 0xFF
};
struct GZip {
u16 signature;
CompressionMethod compressionMethod;
Flags flags;
type::time32_t modificationTime;
ExtraFlags extraFlags;
OperatingSystemID operatingSystemId;
if (flags.FEXTRA) {
u16 extraLength;
u8 extraField[extraLength];
}
if (flags.FNAME) {
char originalFileName[];
}
if (flags.FCOMMENT) {
char comment[];
}
if (flags.FHCRC) {
u16 crc16;
}
u8 data[while($ < std::mem::size() - 8)] [[sealed]];
u32 crc32;
type::Size<u32> isize;
};
#pragma author WerWolv
#pragma description GZip compressed data format
#pragma MIME application/gzip
import type.time;
import type.size;
import std.core;
import std.mem;
using BitfieldOrder = std::core::BitfieldOrder;
bitfield Flags {
FTEXT : 1;
FHCRC : 1;
FEXTRA : 1;
FNAME : 1;
FCOMMENT : 1;
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 8)]];
bitfield ExtraFlags {
padding : 1;
maximumCompression : 1;
fastestCompression : 1;
padding : 5;
};
enum CompressionMethod : u8 {
Reserved = 0 ... 7,
Deflate = 8
};
enum OperatingSystemID : u8 {
FATFileSystem = 0x00,
Amiga = 0x01,
VMS = 0x02,
Unix = 0x03,
VM_CMS = 0x04,
AtariTOS = 0x05,
HPFSFileSystem = 0x06,
Macintosh = 0x07,
ZSystem = 0x08,
CP_M = 0x09,
TOPS_20 = 0x0A,
NTFSFileSystem = 0x0B,
QDOS = 0x0C,
AcordRISCOS = 0x0D,
Unknown = 0xFF
};
struct GZip {
u16 signature;
CompressionMethod compressionMethod;
Flags flags;
type::time32_t modificationTime;
ExtraFlags extraFlags;
OperatingSystemID operatingSystemId;
if (flags.FEXTRA) {
u16 extraLength;
u8 extraField[extraLength];
}
if (flags.FNAME) {
char originalFileName[];
}
if (flags.FCOMMENT) {
char comment[];
}
if (flags.FHCRC) {
u16 crc16;
}
u8 data[while($ < std::mem::size() - 8)] [[sealed]];
u32 crc32;
type::Size<u32> isize;
};
GZip gzip @ 0x00;

441
patterns/hinf_luas.hexpat Normal file
View File

@@ -0,0 +1,441 @@
#pragma author Surasia
#pragma description Halo Infinite HavokScript 5.1 "luas" file
import std.io;
import std.mem;
import std.core;
import std.string;
struct String32 {
s32 length;
if (length != 0) {
char s[length - 1];
u8 endbyte;
}
};
struct String64 {
s64 length;
if (length != 0) {
char s[length - 1];
u8 endbyte;
}
};
enum lua_endian : u8 {
little_endian = 1,
big_endian = 0,
};
enum lua_numbertype : u8 {
_float = 0,
_int32 = 1,
};
struct lua_enum {
s32 value;
String32 string;
};
struct LuaEnum {
s32 count;
lua_enum entries[count];
};
enum HksOpCode : s8 {
GETFIELD,
TEST,
CALL_I,
CALL_C,
EQ,
EQ_BK,
GETGLOBAL,
MOVE,
SELF,
RETURN,
GETTABLE_S,
GETTABLE_N,
GETTABLE,
LOADBOOL,
TFORLOOP,
SETFIELD,
SETTABLE_S,
SETTABLE_S_BK,
SETTABLE_N,
SETTABLE_N_BK,
SETTABLE,
SETTABLE_BK,
TAILCALL_I,
TAILCALL_C,
TAILCALL_M,
LOADK,
LOADNIL,
SETGLOBAL,
JMP,
CALL_M,
CALL,
INTRINSIC_INDEX,
INTRINSIC_NEWINDEX,
INTRINSIC_SELF,
INTRINSIC_LITERAL,
INTRINSIC_NEWINDEX_LITERAL,
INTRINSIC_SELF_LITERAL,
TAILCALL,
GETUPVAL,
SETUPVAL,
ADD,
ADD_BK,
SUB,
SUB_BK,
MUL,
MUL_BK,
DIV,
DIV_BK,
MOD,
MOD_BK,
POW,
POW_BK,
NEWTABLE,
UNM,
NOT,
LEN,
LT,
LT_BK,
LE,
LE_BK,
CONCAT,
TESTSET,
FORPREP,
FORLOOP,
SETLIST,
CLOSE,
CLOSURE,
VARARG,
TAILCALL_I_R1,
CALL_I_R1,
SETUPVAL_R1,
TEST_R1,
NOT_R1,
GETFIELD_R1,
SETFIELD_R1,
NEWSTRUCT,
DATA,
SETSLOTN,
SETSLOTI,
SETSLOT,
SETSLOTS,
SETSLOTMT,
CHECKTYPE,
CHECKTYPES,
GETSLOT,
GETSLOTMT,
SELFSLOT,
SELFSLOTMT,
GETFIELD_MM,
CHECKTYPE_D,
GETSLOT_D,
GETGLOBAL_MEM,
};
enum HksType : u8
{
TNIL,
TBOOLEAN,
TLIGHTUSERDATA,
TNUMBER,
TSTRING,
TTABLE,
TFUNCTION,
TUSERDATA,
TTHREAD,
TIFUNCTION,
TCFUNCTION,
TUI64,
TSTRUCT
};
bitfield REG_CONST_UNUSED {
argB : 17 & 0x1ffff;
argA : 8 & 0xff;
};
bitfield REG_NUM_NUM {
argC : 9 & 0x1ff;
argB : 8 & 0xff;
argA : 8 & 0xff;
};
bitfield REG_NUM_UNUSED {
argB : 8 & 0xff;
argA : 8 & 0xff;
argC : 9 & 0x1ff;
};
bitfield UNUSED_OFFSET_UNUSED {
argA : 8 & 0xff;
argB : 9 & 0x1ff;
argC : 8 & 0xff;
};
bitfield LuaBitfield {
HksOpCode OpCode : 7;
//Arguments : 25; // read on how to process them at https://raw.githubusercontent.com/soupstream/havok-script-tools/main/HavokScriptToolsCommon/HksOpCodes.cs
match(OpCode) {
(HksOpCode::LOADK) : REG_CONST_UNUSED arguments;
(HksOpCode::SETGLOBAL) : REG_CONST_UNUSED arguments;
(HksOpCode::NEWTABLE) : REG_NUM_NUM arguments;
(HksOpCode::CLOSURE) : REG_CONST_UNUSED arguments;
(HksOpCode::RETURN) : REG_NUM_UNUSED arguments;
(HksOpCode::NEWSTRUCT) : REG_NUM_NUM arguments;
(HksOpCode::DATA) : UNUSED_OFFSET_UNUSED arguments;
(_): arguments : 25;
}
};
struct LuaInstruction {
LuaBitfield OpCode;
};
struct LuaConstant {
HksType type;
if (type == HksType::TNIL) {
}
else if (type == HksType::TSTRING) {
String64 string;
}
else if (type == HksType::TNUMBER) {
float number;
}
else if (type == HksType::TBOOLEAN) {
bool boolean;
}
else if (type == HksType::TLIGHTUSERDATA) {
s64 userdata;
}
else {
s64 data;
}
};
struct HksLocal {
String64 string;
u32 start;
u32 end;
};
struct LuaFunction {
s32 upvaluecount;
s32 paramcount;
s8 isVarArg;
s32 slotCount;
s32 unk;
s32 instruction_count;
if ($ % 4 != 0) {
$ += (4 - ($ % 4));
}
LuaInstruction instructions[instruction_count];
s32 constantCount;
LuaConstant constants[constantCount];;
s32 HasDebugInfo;
if (HasDebugInfo != 0) {
u32 LineCount;
u32 LocalsCount;
u32 UpValueCount;
u32 LineBegin;
u32 LineEnd;
String64 path;
String64 functionName;
s32 Lines[LineCount];
HksLocal Locals[LocalsCount];
String64 UpValues[UpValueCount];
}
s32 function_count;
LuaFunction children[function_count];
};
struct LuaStructHeader {
String64 name;
s32 structID;
s32 pad;
s24 unk1;
HksType type;
s32 unk2;
};
struct LuaStructMember {
s32 pad;
LuaStructHeader header;
s32 index;
};
struct LuaStruct {
try {
u32 unk;
LuaStructHeader ExtendedStructHeader;
s64 ExtendedStructMemberCount;
LuaStructMember ExtendedStructMembers[ExtendedStructMemberCount];
u32 unk2;
} catch {
std::print("WARNING: this file does not support structs.");
}
try {
LuaStructHeader StructHeader;
s64 StructMemberCount;
LuaStructMember StructMembers[StructMemberCount];
} catch {
std::print("WARNING: this file does not support multiple structs.");
}
};
struct LuaHeader {
char luaheader[4];
u8 version;
u8 format;
lua_endian Endianness;
if (Endianness == lua_endian::big_endian) {
std::core::set_endian(std::mem::Endian::Big);
} else {
std::core::set_endian(std::mem::Endian::Little);
}
u8 intSize;
u8 size_tSize;
u8 instruction_size;
u8 number_size;
lua_numbertype numberType;
u8 flags;
u8 unknown;
LuaEnum enums;
LuaFunction LuaFunctions;
LuaStruct LuaStructures;
};
struct Header {
u32 magic;
u32 version;
u64 RootStructAltGUID;
u64 AssetChecksum;
u32 DependencyCount;
s32 DataBlockCount;
s32 TagStructCount;
s32 DataReferenceCount;
s32 TagReferenceCount;
u32 StringTableSize;
u32 ZoneSetDataSize;
u32 Unk_0x34;
u32 HeaderSize;
u32 DataSize;
u32 ResourceDataSize;
u32 ActualResoureDataSize;
u8 HeaderAlignment;
u8 TagDataAlignment;
u8 ResourceDataAlignment;
u8 ActualResourceDataAlignment;
s32 IsResource;
};
struct _s_tagblock {
char tagblock[16];
s32 count;
};
struct _s_resource {
char resource[12];
s32 count;
};
struct _s_data {
char data[24];
};
struct Data_Block {
u32 Size;
u16 Padding;
u16 Section;
u64 Offset;
};
struct Tag_Def_Structure {
s64 GUID_1;
s64 GUID_2;
u16 Type;
u16 Unk_0x12;
s32 TargetIndex;
s32 FieldBlock;
u32 FieldOffset;
};
struct Tag_Dependency {
char GroupTag[4] [[format("std::string::reverse")]];
u32 NameOffset;
u64 AssetID;
u32 GlobalID;
u32 Unk_0x14;
};
struct Data_Reference {
s32 ParentStructIndex;
s32 Unk_0x04;
s32 TargetIndex;
s32 FieldBlock;
u32 FieldOffset;
};
struct _s_tagref {
s64 GlobalHandle;
s32 ref_id_int;
s32 ref_id_sub;
s32 ref_id_center_int;
char TagGroupRev[4] [[format("std::string::reverse")]];
s32 local_handle;
};
struct Tag_Fixup_Reference {
s32 FieldBlock;
u32 FieldOffset;
u32 NameOffset;
s32 DependencyIndex;
_s_tagref tagreference @ header.HeaderSize + datablocks[FieldBlock].Offset + FieldOffset;
};
struct InternalStruct {
s64 vtablespace;
u32 global_tag_id;
char local_tag_handle[4];
};
struct LuaReferencedTagContainer {
_s_tagref tagReference;
};
struct LuaScriptTagDefinition {
u32 luaFileName;
_s_data luaFileData;
char luaFileNameString[256];
_s_tagblock referencedTags_count;
LuaHeader lua_header;
try {
char pad[14];
LuaReferencedTagContainer referencedTags[referencedTags_count.count];
} catch {
std::print("This file does not support referenced Tags.");
}
};
// TAG LAYOUT
Header header @ 0x00;
Tag_Dependency Dependencies[header.DependencyCount] @ 0x50;
Data_Block datablocks[header.DataBlockCount] @ header.DependencyCount * 0x18 + 0x50;
Tag_Def_Structure tagdefstructure[header.TagStructCount] @ header.DependencyCount * 0x18 + 0x50 + header.DataBlockCount * 0x10;
Data_Reference Data_References[header.DataReferenceCount] @ header.DependencyCount * 0x18 + 0x50 + header.DataBlockCount * 0x10 + header.TagStructCount * 0x20;
Tag_Fixup_Reference tagfixupreference[header.TagReferenceCount] @ header.DependencyCount * 0x18 + 0x50 + header.DataBlockCount * 0x10 + header.TagStructCount * 0x20 + header.DataReferenceCount * 0x14;
char ZoneSet[header.ZoneSetDataSize] @ header.HeaderSize - header.ZoneSetDataSize;
InternalStruct internalstruct @ header.HeaderSize;
LuaScriptTagDefinition lua_script @ header.HeaderSize + 16;

View File

@@ -0,0 +1,77 @@
#pragma author Surasia
#pragma description Halo Infinite Module
#pragma array_limit 4294967295
#pragma pattern_limit 4294967295
import std.string;
struct ModuleHeader
{
char magic[4];
s32 version;
s64 moduleId;
u32 fileCount;
s32 manifest0Count;
s32 manifest1Count;
s32 manifest2Count;
s32 resourceIndex;
u32 stringSize;
u32 resourceCount;
u32 blockCount;
u64 buildVersion;
s64 hd1Delta;
u64 dataSize;
};
struct ModuleFile
{
u32 resourceCount;
s32 parentIndex;
s16 pad;
u16 blockCount;
s32 blockIndex;
s32 resourceIndex;
char class[4] [[format("std::string::reverse")]];
u64 dataOffsetBytes;
u64 dataOffset = dataOffsetBytes & 0x0000FFFFFFFFFFFF [[export]];
u16 dataOffsetFlags = dataOffsetBytes >> 48 [[export]];
u32 totalCompressedSize;
u32 totalUncompressedSize;
u32 globalTagId;
u32 uncompressedHeaderSize;
u32 uncompressedTagDataSize;
u32 uncompressedResourceDataSize;
u32 uncompressedActualResourceDataSize;
u32 resourceBlockCount;
u32 nameOffset;
s32 parentResource;
u64 assetChecksum;
u64 assetId;
};
struct ModuleBlock
{
u32 compressedOffset;
u32 compressedSize;
u32 decompressedOffset;
u32 decompressedSize;
bool compressed;
s24 pad;
};
struct Module
{
ModuleHeader module_header;
ModuleFile files[module_header.fileCount];
$ = $ + 8;
s32 resources[module_header.resourceCount];
ModuleBlock blocks[module_header.blockCount];
padding[while($[$] == 0)];
std::print("Compressed Tag Data starts at: {}", $);
};
Module module_root @ 0x00;

168
patterns/hinf_tag.hexpat Normal file
View File

@@ -0,0 +1,168 @@
#pragma author Surasia
#pragma description Halo Infinite Tag File
import std.string;
import type.guid;
u32 fileStart = 0;
enum TagSectionType : u16
{
header,
tagData,
resourceData
};
enum TagStructType : u16
{
mainStruct,
tagBlock,
resource,
custom,
literal
};
struct TagHeader
{
char magic[4];
s32 version;
u64 hash;
u64 checksum;
u32 dependencyCount;
u32 dataBlockCount;
u32 tagStructCount;
u32 dataReferenceCount;
u32 tagReferenceCount;
u32 stringTableSize;
u32 zoneSetDataSize;
u32 unknown0;
u32 headerSize;
fileStart = headerSize;
u32 dataSize;
u32 resourceDataSize;
u32 actualResourceDataSize;
u8 headerAlignment;
u8 tagDataAlignment;
u8 resourceDataAlignment;
u8 actualResourceAlignment;
u32 unknown1;
};
struct TagDependency
{
char tagGroup[4] [[format("std::string::reverse")]];
u32 nameOffset;
u64 assetId;
u32 globalId;
u32 parentTag;
};
struct TagDataBlock
{
u32 entrySize;
u16 pad;
TagSectionType section;
u64 offset;
if (section == TagSectionType::tagData)
{
char dataBlock[entrySize] @ fileStart + offset;
}
};
struct TagStruct
{
type::GUID guid;
TagStructType type;
u16 unknown0;
s32 targetIndex;
s32 fieldBlockIndex;
u32 fieldOffset;
};
struct TagDataReference
{
s32 parentStructIndex;
s32 unknown0;
s32 targetIndex;
s32 fieldBlockIndex;
u32 fieldOffset;
};
struct TagReference
{
s32 fieldBlockIndex;
u32 fieldOffset;
u32 nameOffset;
s32 dependencyIndex;
};
struct TagZonesetHeader
{
s32 version;
u32 zonesetCount;
u32 footerCount;
s32 parentCount;
};
struct TagZonesetInstanceHeader
{
s32 stringId;
u32 tagCount;
u32 parentCount;
u32 footerCount;
};
struct TagZonesetTag
{
u32 globalId;
s32 stringId;
};
struct TagZonesetInstance
{
TagZonesetInstanceHeader header;
TagZonesetTag tags[header.tagCount];
TagZonesetTag footers[header.footerCount];
s32 parents[header.parentCount];
};
struct TagZoneset
{
TagZonesetHeader zoneset_header;
TagZonesetInstance instances[zoneset_header.zonesetCount];
};
struct Tag
{
TagHeader header;
TagDependency dependencies[header.dependencyCount];
TagDataBlock data_blocks[header.dataBlockCount];
TagStruct tag_structs[header.tagStructCount];
TagDataReference data_references[header.dataReferenceCount];
TagReference tag_references[header.tagReferenceCount];
TagZoneset zonesets;
if (header.headerSize != $)
{
std::print("Header size is wrong, additional data in tag");
$ = header.headerSize;
std::print("Data offset now at: {}", $);
}
};
Tag tag_root @ 0x00;

View File

@@ -1,6 +1,9 @@
#pragma author WerWolv
#pragma description Icon (.ico) or Cursor (.cur) files
#pragma endian little
#include <std/sys.pat>
import std.sys;
#pragma MIME image/vnd.microsoft.icon
#pragma MIME image/x-icon

Some files were not shown because too many files have changed in this diff Show More