Compare commits

...

85 Commits

Author SHA1 Message Date
WerWolv
5359e385ea git: Added build folders to gitignore 2025-09-06 07:33:32 +02:00
Nik
75bcb487ee patterns/ico: Fixed duplicate variable name 2025-09-01 22:22:43 +02:00
Ivy Fan-Chiang
fb214600ec patterns/ico: Add embedded BMP and PNG parsing to ICO pattern (#426)
Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-31 14:32:21 +02:00
Kris Dekeyser
d95390ea42 patterns/jpeg: added support for extra data in APP0 section (#417)
Apple Multi-Picture Format JPEGs often have 4 extra bytes in the APP0 section. The pattern now skips any extra bytes beyond the fixed APP0 data.

Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-31 11:40:27 +02:00
Marc Jones
284ca8d325 patterns: Add UEFI Firmare Volume Variable Store pattern (#421)
* Add UEFI Firmare Volume Variable Store pattern

Add a pattern for UEFI Firmare Volume Variable store.
This file type is commonly used with virtual machine UEFI variable files,
like OVMF.fd used with QEMU. You could also extract a UEFI firmware binary
from a flash device, search for the FV Variable Store, and set this pattern
to the FV address.

Signed-off-by: Marc Jones <marcj303@gmail.com>

* Fixed description pragma

---------

Signed-off-by: Marc Jones <marcj303@gmail.com>
Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-31 11:38:32 +02:00
ODeux
6630180276 patterns: Add .NET BinaryFormatter pattern (#416)
* Add dotnet BinaryFormatter pattern

* Add dotnet BinaryFormatter test

* Update README.md

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-31 11:37:59 +02:00
Henri Asseily
ff68d1e23d patterns: Added Apple IIGS SHR + SHR 3200 + SHR PWA Animation pattern (#432)
* Added SHR pattern

* Added IIGS SHR animation test file

* Added pattern to readme

* Added description and author

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-31 11:36:00 +02:00
Tom Arrow
70dd55aa6b patterns/q3demo: Quake 3 demos: Handle corrupted files more gracefully (#414)
Quake 3 demos: Basic safeguard against corrupted files

Co-authored-by: Tom <tomarrowtomarrow@hotmail.com>
2025-08-31 11:31:20 +02:00
DmitriLeon2000
76f850c543 patterns/fas/was: Update FAS, WAS/WA3 pattern files and README.md (#425)
* Add .fas and .was pattern files (Oska DeskMates)

* Update .was pattern file

* Update .was/.wa3 pattern file

* Update README.md

* Update README.md

* Update .fas & .was pattern files

* Update README.md

* Update fas_oskasoftware_old.hexpat

* Added WAS test file

* Update WAS test file

* Update was_oskasoftware.hexpat

* Update was_oskasoftware.hexpat

* Update fas_oskasoftware_old.hexpat

* Update fas_oskasoftware.hexpat

* Update README.md

Replacing backward slashes with forward ones in the `WAS` row.

* Update fas_oskasoftware_old.hexpat

* Update was_oskasoftware.hexpat
2025-08-31 11:28:45 +02:00
MicroBlock
74c06b74f7 patterns/lua53: Fix lua53 long string (#427)
Update lua53.hexpat
2025-08-31 11:27:59 +02:00
Vemmi
f13d9d3894 patterns/upk-ue3: Add magic pragma (#436) 2025-08-31 11:23:09 +02:00
mjarduk
c4c75a9ab2 patterns: Add a pattern for Roblox .pack files (#435)
* Added the .pack format

* Clarified some fields by changing the naming
2025-08-25 22:22:32 +02:00
Nik
7278a22eb2 includes/std: Fix MagicSearch implementation not being found 2025-08-24 13:34:47 +02:00
Nik
91fd36097c includes/std: Fix use of std::mem::size and std::mem::base_address in library 2025-08-24 12:04:33 +02:00
Nik
afffd7eced includes/std: Added section parameters to a few std::mem functions 2025-08-24 12:00:05 +02:00
Lexi Mayfield
7fd79ec9fd patterns: Added AppleSingle, AppleDouble, CHD, TARC patterns (#431)
* Commit patterns I've collected

- AppleSingle/AppleDouble pattern, used for macOS resource forks.
- MAME CHD file format, currently only supports v5.
- KEX Engine proprietary TARC format, used by various Nightdive games.

* Add to README
2025-08-20 19:32:11 +02:00
Mark Jansen
6b9f39cc21 patterns: Add SDB pattern (#424)
Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-15 17:16:50 +02:00
Stephen Hewitt
9207282bcf patterns: Added Commodore BASIC (#428)
* Commodore BASIC

* Update desc

* I made it

* Implemented suggestion

* Implemented suggestion

* Test file

* Rename file

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2025-08-02 20:13:35 +02:00
WerWolv
5ed64f9f08 tests: Include patterns folder as include path to support import * from X as Y
#426
2025-08-02 19:29:16 +02:00
WerWolv
a75a7a5b98 git: Let the CI still work correctly in PRs 2025-07-29 22:26:27 +02:00
WerWolv
60c8d93449 git: Make sure other repos check out the right repo 2025-07-29 21:57:54 +02:00
WerWolv
d3b05fd753 git: Allow test action to be imported by other projects 2025-07-29 21:38:20 +02:00
paxcut
3b2f098b09 fix: fixed two typos in this theme (#423) 2025-07-18 08:39:37 -07:00
paxcut
f4f004f0eb improv: Added new text editor semantic highlighting themes. (#422)
* improv: Added new text editor semantic highlighting themes.

Current themes use the same colors  for all identifier types. These two new themes are just examples of what the new feature makes possible.
One aims to simulate CLion Darcula theme and the other is an extension of the original dark theme.

* fix: corrected entries to readme file

* fix: corrected base for the theme

Made sure readme looks correctly on fork and links go to files.
2025-07-16 07:01:43 -07:00
paxcut
5c6cb9dccc fix: issues with two remaining themes. (#420)
Both one-dark and nocturne used global-doc-comment instead of the correctly named doc-global-comment. Typos in Nocturned didn't prevent the theme from loading, but left annoying messages in the log. As explained by the author, the base color map must be one of the three that imhex loads by default, dark,light and/or classic. Even if that is not the case, the name given in base must be that of a valid color map that has been loaded prior to the ones that use it as a base.
2025-07-11 03:42:45 -07:00
paxcut
44717e9b19 fix: incorrect names and values for some colormap entries of the text editor. (#419)
fix: incorrect names and values for some colormap entries of the test editor.

Some names were changed from the original that need to be changed back to make the old colormaps compatible with the new system.
Also, the colors of the console text were incorrectly set to be equal to each other.
This Pr brings back the old names and hopefully fixes problems of incorrect color being used
2025-07-11 02:25:02 -07:00
Dominik Tamiołło
bc35349e0f PEF - fix export count calculation (#418)
* Initial version of PEF

* add pef test file

* Fixed export hash slot count calculation

---------

Co-authored-by: paxcut <53811119+paxcut@users.noreply.github.com>
2025-07-08 15:21:07 -07:00
Nik
fed5db4109 Add Capcom's Devil May Cry 3 HD .mod hexpat (#415)
* Add Capcom's Devil May Cry 3 HD .mod hexpat

Hex Pattern file for Capcom's Devil May Cry 3 HD Collection's .mod (3D Models) files

* Update DMC3 HD Mod.hexpat

* Update DMC3 HD Mod.hexpat

* Update DMC3 HD Mod.hexpat

* Add files via upload

* Update README.md

* Rename DMC3 HD Mod.hexpat to dmc3_hd_mod.hexpat

* Delete patterns/dmc3_hd_mod.hexpat

* Delete tests/patterns/test_data/dmc3_hd_mod.hexpat.mod

* Add files via upload

* Update dmc3_hd_mod.hexpat

---------

Co-authored-by: haruse23 <mrjokeromar123@outlook.com>
2025-07-08 14:51:54 -07:00
klirktag
1d41392215 Add support for medium and large mp4 files (#413)
add support for medium and large mp4
2025-06-29 07:33:48 -07:00
ruokeqx
0b75336638 feat(pcapng): add support for Decryption Secrets Block (#411)
feat(pcapng): add support for Decryption Secrets Block
2025-06-19 05:13:43 -07:00
C3pa
e3edbd5a6f Small documentation improvements (#408)
doc: add syntax highlighting to std::ptr::NullablePtr example, fix description of type::escape_bytes

Co-authored-by: paxcut <53811119+paxcut@users.noreply.github.com>
2025-06-17 23:55:51 -07:00
C3pa
b10a37af67 Docs: describe how to create multidimensional std::Array (#407)
doc: descirbe how to create multidimensional std::Array

Co-authored-by: paxcut <53811119+paxcut@users.noreply.github.com>
2025-06-17 23:46:33 -07:00
Dominik Tamiołło
103d434cc5 Add PEF (Preffered Executable Format) support (#406)
* Initial version of PEF

* add pef test file

---------

Co-authored-by: paxcut <53811119+paxcut@users.noreply.github.com>
2025-06-17 22:24:54 -07:00
Fabian Neundorf
7716b9d6e7 patterns: Add support for smk (#399)
* patterns: Add support for smk

* patterns: Use builtin function and separate SMK struct

---------

Co-authored-by: paxcut <53811119+paxcut@users.noreply.github.com>
2025-06-17 14:57:36 -07:00
Tom Arrow
ad1e300674 Quake 3 Engine demo pattern & test file (#402)
* Quake 3 Engine demo pattern & test file

* Quake 3 Engine demo: Increase limits (demo files can be big) and additional condition

* Quake 3 demo specify little endian

* Quake 3 demo format: Add message type detection

* Quake 3 demo format: Read serverTime for snapshot messages

* Quake 3 demo pattern: fixed bug/typo/sleepy coding

* Quake 3 demo pattern: Ability to read CS_SERVERINFO

* Quake 3 demo pattern: Read CS_SYSTEMINFO as well

* Quake 3 demo pattern: Read first serverCommand per message

* Added Quake 3 engine demo pattern to readme

* Change Quake 3 engine demo entry in readme table

---------

Co-authored-by: Tom <tomarrowtomarrow@hotmail.com>
2025-06-17 02:06:24 -07:00
zn123
e918ce52b9 Add flv.hexpat (#401)
* Add flv.hexpat

https://veovera.org/docs/enhanced/enhanced-rtmp-v2#flv-file-format-overview

* test flv.hexpat

* Update README.md

add flv patterns

* Update flv.hexpat

Optimize spaces

* Update flv.hexpat

fix
2025-06-16 20:25:54 -07:00
Nik
db4d62aa20 patterns/fs: Refactor all partition types into a FS module 2025-05-25 19:51:32 +02:00
WerWolv
d96bfbb942 tests: Execute and test format functions 2025-05-25 12:43:21 +02:00
Nik
2070c95f58 yara: Added crypto identification rules 2025-05-24 23:56:56 +02:00
Nik
877895741d includes/hex: Fixed Json types being twice the size they need to be 2025-05-17 19:45:30 +02:00
Nik
6b279b8375 patterns/exfat: Fixed eval depth error 2025-05-17 16:07:11 +02:00
Khoo Hao Yit
a692b22ecc patterns: Add support for exFAT (#398) 2025-05-17 13:13:35 +02:00
Claudius Coenen
2ae0499293 patterns/wintec_tes: Slightly nicer formatting in a list of entries (#397)
slightly nicer formatting in a list of entries
2025-05-16 20:25:04 +02:00
thrasher
bd06987e8e patterns/zip: Add zip64 support (#395)
Co-authored-by: Nik <werwolv98@gmail.com>
2025-05-15 21:22:20 +02:00
Fabian Neundorf
74e08623f1 patterns: Add support for flc/flic files (#396)
* patterns: Add support for flc/flic files

* patterns: Add #pragma description for flc

---------

Co-authored-by: Nik <werwolv98@gmail.com>
2025-05-15 20:17:13 +02:00
Nik
a860c396fa patterns: Added Java HPROF pattern 2025-05-15 17:23:06 +02:00
Nik
da934e2045 patterns/optee_ta: Fixed missing author and description pragmas 2025-05-15 08:07:13 +02:00
Nik
394ef80611 patterns: Added OPTEE Trusted Application format 2025-05-13 22:17:05 +02:00
Grant Kim
2ce182b1b3 patterns/tiff: Tags for DNG and TIFF extensions (#376)
Add Tags for DNG and TIFF extensions

References: https://www.loc.gov/preservation/digital/formats/content/tiff_tags.shtml
https://helpx.adobe.com/camera-raw/digital-negative.html

Co-authored-by: Nik <werwolv98@gmail.com>
2025-05-10 13:59:22 +02:00
Glenn Hartmann
6cadad3d1f pattern/protobuf: Allow parsing of nested messages (#378)
Mostly this enables attempted recursive parsing of submessages. Note
that it is inherently impossible to determine the underlying data type
for LengthDelimited for sure, so this is a best-effort attempt. The user
can disable recursive submessage parsing via Settings.

Other minor changes:
* added #pragma MIME and #pragma endian directives
* enabled UTF-8 display for LengthDelimited types
* added signed LEB128 display for Varint types (although this doesn't
  seem to be working on my test case)
* swapped if/else-if structure for match
* fail upon receiving unknown or unsupported WireType

Co-authored-by: Nik <werwolv98@gmail.com>
2025-05-10 13:53:01 +02:00
David Schneider
7ad9cd4f41 patterns: Added support for GNU MO files (#383) 2025-05-10 11:49:02 +00:00
0xZ3R0
bcaeef31d7 pattern: Added DJI Firmware Pattern (#392)
* added IM*H pattern

* improved README
2025-05-10 13:35:14 +02:00
Niketin
ee8d2f50b7 pattern/bencode: Various fixes (#393)
* Fix bencode dictionary

When parsing a bencode dictionary, the end character 'e' was never
consumed. This caused a misinterpretation of the character as struct Value of
an unknown type 'e'.

* Fix bencode list

A list was not included in the Value's parsing logic so it may have
been mistakenly parsed as a string.

* Fix std::ctype::isprint not including space

The space character, 0x20, is considered as a printable character in ASCII
and in >=C89. Adding it to the range of std::ctype::isprint also fixes
other std::ctype functions that use it.

* Fix bencode byte string formatting

Byte strings do not render nicely in pattern data's value column if they
contain non-printable characters. This commit makes the value of byte
strings to be surrounded by quotation marks, and renders a warning text
without quotation marks if the byte string contains non-printable
characters.
2025-05-10 13:34:36 +02:00
DEREFERENC3D
5b8dde19a9 patterns: Added definition for the OpenRGB profile format (#387)
patterns: Add definition for the OpenRGB profile format (up to version 5)
2025-05-10 13:34:16 +02:00
Dhash Shrivathsa
0b0eff0cb6 pattern/macho: Load Commands should include additional specializations (LoadDyLib) (#386)
* add additional command specialization

* address PR feedback
2025-05-10 13:34:01 +02:00
Enaium
3185503be1 patterns: Added valve vpk pattern (#384) 2025-05-10 10:29:50 +00:00
Sabhya
3a64207e58 pattern: Added pcapng capture file pattern (#385)
add: pcapng hexpat
2025-05-10 12:26:39 +02:00
bluisblu
c94d42d5c0 patterns: Added Radcore Cement v1.2 file pattern (#382)
* Add Radcore Cement file pattern

* Update pattern list in README.md
2025-05-10 11:57:51 +02:00
Caleb Denio
45c4c1d35b patterns/uf2: Added magic number (#379) 2025-05-10 11:54:29 +02:00
Jean Pereira
f93be1ea06 themes: Added Nocture theme (#377)
minimal dark theme based on Dracula but more darker
2025-05-10 11:53:28 +02:00
Shadlock0133
5ed9c0fd4d patterns/elf: use Rel{a} structs for parsing REL{A} section data (#375) 2025-05-10 11:52:35 +02:00
C3pa
11d373319f pattern/pe: Check for imageNameRVA in pe.hexpat before reading imageName (#372) 2025-05-10 11:52:00 +02:00
Jonathan Ostrus
e7d366571d pattern/pex: Added formatter for opcode and some other name cleanups (#371) 2025-05-10 11:51:43 +02:00
Caleb Denio
b6df1742b6 patterns: Add ZIM pattern (#367)
* Add ZIM pattern file

* Add test file

* namespace!
2025-05-10 11:51:04 +02:00
Jake Ryan
0b9e83ff8a patterns: Add support for MagicaVoxel .vox files (#390)
* patterns: Added support for MagicaVoxel .vox files

* Fixed incorrect IMAP field size and added basic chunk size mismatch detection and recovery.

* Fixed pattern for "_r" values and added RotationToMat3.

* Added test vox file.

---------

Co-authored-by: paxcut <53811119+paxcut@users.noreply.github.com>
2025-04-30 08:55:53 -07:00
paxcut
c0fc748de6 fix: rgb color attribute is always blue.
The color attribute does not nclude an alpha component but the RGB types were including an alpha of 255 in their color attributes. The 0xFF was being assigned to the blue component hence the error.
2025-04-30 08:40:32 -07:00
Hrant
89307ba8a5 added NTFS pattern (#391)
Thank you!

* added NTFS pattern

* improved the PR

* remove test-data & fixed sd

---------

Co-authored-by: Hrant Tadevosyan <Hrant.Tadevosyan@connectwise.com>
2025-04-30 04:00:52 -07:00
ZHANG Yuntian
a12b5ba406 git: Fix table formatting (#374)
Signed-off-by: ZHANG Yuntian <yt@radxa.com>
2025-03-24 07:33:01 +01:00
ZHANG Yuntian
537ce67895 patterns/partition_table: Added basic MBR and GPT patterns (#358)
Both are conbined into a single pattern due to how GPT always has a pseudo MBR to maintain backward compatibility.
2025-03-24 07:15:25 +01:00
Scrivener07
1771c1f077 patterns: Added support for PEX Papyrus executables (#370)
Added support for PEX, Bethesda's Papyrus executable for compiled script files.
- Skyrim
- Fallout 4
- Fallout 76
- Starfield

Co-authored-by: Jonathan Ostrus <12855515+jbostrus@users.noreply.github.com>
2025-03-23 11:30:43 +01:00
Caleb
cef20e24a7 patterns/zip: Fix 0x5455 extended timestamp fields crash, improve date/time presentation (#363)
Co-authored-by: Nik <werwolv98@gmail.com>
2025-03-22 13:57:42 +01:00
Andreas Wallner
8e7cfd9442 patterns: Add STDFv4 pattern (#326) 2025-03-22 13:54:50 +01:00
5ec1cff
d0ba754dc2 patterns/zip: Handling padded extra fields (#369)
* Handling unresolved extra fields

* Update zip.hexpat

* Update zip.hexpat

* Update zip.hexpat
2025-03-22 13:54:26 +01:00
mheimlich
9f92c38ecf patterns: Add ADTFDAT pattern (#368) 2025-03-22 13:52:59 +01:00
Geky
0844e07056 patterns: Added DS Cartridge Header pattern, updated pyc versions (#366)
* add support for cartridge size type $54

Added support for cartridge size type $54, corresponding to 1.5 MiB (96 banks).

* add missing license

* Add GBA Cartridge Header

* Update README.md

Added GBA information to README.md and corrected a typo.

* Add DS Cartridge Header

* Update ReadMe.md

* Update pyc.hexpat

Included additional versions
2025-03-22 13:51:46 +01:00
C3pa
66fc006b08 includes/std: Update hex::dec::zlib_decompress description (#364)
docs: Update hex::dec::zlib_decompress description
2025-03-22 13:51:00 +01:00
Eddy S.
5bc66df14f patterns/elf: Set endianness globally (#362) 2025-03-22 13:50:17 +01:00
MrMcX
7310a10214 patterns: Add pattern for microsoft prefetch files (uncompressed SCCA format only) (#361)
* Add pattern for microsoft prefetch files (uncompressed SCCA format only)

* Update pf.hexpat

Add longer explanation
2025-03-22 13:49:50 +01:00
MrMcX
34ee3107e2 patterns: Implement Windows thumbcache pattern (#359)
* Implement Windows thumbcache pattern

* Include thumbcache pattern in README.md

* Update thumbcache.hexpat

- Use English warning text
- handle unknown file version
- add thanks to joachimmetz
- create virtual file
- use more elegant magic type

* Update thumbcache.hexpat

- Use padding data type for paddings

* Update thumbcache.hexpat

- make pattern more robust against single erroneous cache record
2025-03-22 13:48:57 +01:00
Shadlock0133
c4378ffb14 patterns/elf: Use e_shstrndx to get section names instead of guessing (#357) 2025-03-22 13:47:43 +01:00
eli_w
5ad7f0c1e7 patterns/ani: Improved compatibility with error size defined in ani files (#354) 2025-03-22 13:46:49 +01:00
Mrmaxmeier
df97fc7257 patterns/includes: More misc cleanups (#353)
* patterns/zip: simplify find_sequence_in_range check

hex(340282366920938463463374607431768211455) =
0xffffffffffffffffffffffffffffffff

* patterns/7z: use Magic<> for signature, add pragma

* patterns/{nro, ogg}: use Magic<> for signature

* patterns/ttf: refactor, check magic

Use the "Field field @ position;" syntax instead of saving and restoring
the cursor position.

* readme: fix copy-paste error in pattern listing
2025-03-22 13:46:06 +01:00
Vemmi
3ad263783d patterns: Added Unreal Engine 3 UPK pattern (#352)
* patterns: Add unreal 3 upk pattern

* readme: Add link to upk-ue3.hexpat
2025-03-22 13:45:25 +01:00
Ilya Sorochan
5ccd431320 patterns: Added RPM pattern (#350) 2025-03-22 13:44:55 +01:00
Nik
fb8e5e3f77 disassemblers/jvm: Fixed mask and mnemonic of areturn and athrow 2025-02-28 11:42:44 +01:00
108 changed files with 11443 additions and 323 deletions

View File

@@ -7,6 +7,7 @@ on:
branches: [ '*' ]
repository_dispatch:
types: [run_tests]
workflow_call:
jobs:
tests:
@@ -21,6 +22,7 @@ jobs:
- name: 🧰 Checkout
uses: actions/checkout@v4
with:
repository: ${{ github.repository_owner }}/ImHex-Patterns
submodules: recursive
- name: ⬇️ Install dependencies

1
.gitignore vendored
View File

@@ -2,6 +2,7 @@
tests/cmake*/
tests/build*/
build/
cmake-build-*/
.vscode/
.devcontainer/

View File

@@ -25,9 +25,12 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
|------|------|------|-------------|
| 3DS | | [`patterns/3ds.hexpat`](patterns/3ds.hexpat) | Autodesk 3DS Max Model file |
| 7Z | | [`patterns/7z.hexpat`](patterns/7z.hexpat) | 7z File Format |
| ADTFDAT | | [`patterns/adtfdat.hexpat`](patterns/adtfdat.hexpat) | [ADTFDAT files](https://digitalwerk.gitlab.io/solutions/adtf_content/adtf_base/adtf_file_library) |
| ADTS | `audio/x-hx-aac-adts` | [`patterns/adts.hexpat`](patterns/adts.hexpat) | ADTS/AAC audio files |
| AFE2 | | [`patterns/afe2.hexpat`](patterns/afe2.hexpat) | Nintendo Switch Atmosphère CFW Fatal Error log |
| ANI | `application/x-navi-animation` | [`patterns/ani.hexpat`](patterns/ani.hexpat) | Windows Animated Cursor file |
| AppleSingle | `application/applefile` | [`patterns/apple_single_double.hexpat`](patterns/apple_single_double.hexpat) | AppleSingle Dual Fork file |
| AppleDouble | `multipart/appledouble` | [`patterns/apple_single_double.hexpat`](patterns/apple_single_double.hexpat) | AppleDouble Resource Fork/Finder Metadata file |
| AR | `application/x-archive` | [`patterns/ar.hexpat`](patterns/ar.hexpat) | Static library archive files |
| ARC | | [`patterns/arc.hexpat`](patterns/arc.hexpat) | Minecraft Legacy Console Edition ARC files |
| ARIA2 | | [`patterns/aria2.hexpat`](patterns/aria2.hexpat) | ARIA2 Download Manager Control files |
@@ -43,12 +46,13 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| BSON | `application/bson` | [`patterns/bson.hexpat`](patterns/bson.hexpat) | BSON (Binary JSON) format |
| bplist | `application/x-bplist` | [`patterns/bplist.hexpat`](patterns/bplist.hexpat) | Apple's binary property list format (bplist) |
| BSP | | [`patterns/bsp_goldsrc.hexpat`](patterns/bsp_goldsrc.hexpat) | GoldSrc engine maps format (used in Half-Life 1) |
| BZIP3 | | [`patterns/bzip3.hexpat`](patterns/bzip3.hexpat) | GoldSrc engine maps format (used in Half-Life 1) |
| BZIP3 | | [`patterns/bzip3.hexpat`](patterns/bzip3.hexpat) | Parses BZip3 compression (file format) by Kamila Szewczyk |
| CAB | | [`patterns/cab.hexpat`](patterns/cab.hexpat) | Microsoft Cabinet (CAB) Files |
| CCHVA | | [`patterns/cchva.hexpat`](patterns/cchva.hexpat) | Command and Conquer Voxel Animation |
| CCVXL | | [`patterns/ccvxl.hexpat`](patterns/ccvxl.hexpat) | Command and Conquer Voxel Model |
| CCPAL | | [`patterns/ccpal.hexpat`](patterns/ccpal.hexpat) | Command and Conquer Voxel Palette |
| CDA | | [`patterns/cda.hexpat`](patterns/cda.hexpat) | Compact Disc Audio track |
| CHD | | [`patterns/chd.hexpat`](patterns/chd.hexpat) | MAME Compressed Hunks of Data file |
| CHM | `application/vnd.ms-htmlhelp` | [`patterns/chm.hexpat`](patterns/chm.hexpat) | Windows HtmlHelp Data (ITSF / CHM) |
| COFF | `application/x-coff` | [`patterns/coff.hexpat`](patterns/coff.hexpat) | Common Object File Format (COFF) executable |
| CPIO | `application/x-cpio` | [`patterns/cpio.hexpat`](patterns/cpio.hexpat) | Old Binary CPIO Format |
@@ -59,6 +63,7 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| DICOM | `application/dicom` | [`patterns/dicom.hexpat`](patterns/dicom.hexpat) | DICOM image format |
| DMG | | [`patterns/dmg.hexpat`](patterns/dmg.hexpat) | Apple Disk Image Trailer (DMG) |
| DMP | | [`patterns/dmp64.hexpat`](patterns/dmp64.hexpat) | Windows Kernel Dump(DMP64) |
| DOTNET_BinaryFormatter | | [`patterns/dotnet_binaryformatter.hexpat`](patterns/dotnet_binaryformatter.hexpat) | .NET BinaryFormatter |
| DPAPI_Blob | | [`patterns/dpapblob.hexpat`](patterns/dpapiblob.hexpat) | Data protection API Blob File Format |
| DPAPI_MasterKey | | [`patterns/dpapimasterkey.hexpat`](patterns/dpapimasterkey.hexpat) | Data protection API MasterKey |
| DS_Store | | [`patterns/dsstore.hexpat`](patterns/dsstore.hexpat) | .DS_Store file format |
@@ -66,13 +71,17 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| DTED | | [`patterns/dted.hexpat`](patterns/dted.hexpat) | Digital Terrain Elevation Data (DTED) |
| ELF | `application/x-executable` | [`patterns/elf.hexpat`](patterns/elf.hexpat) | ELF header in elf binaries |
| EVTX | `application/x-ms-evtx` | [`patterns/evtx.hexpat`](patterns/evtx.hexpat) | MS Windows Vista Event Log |
| EXT4 | | [`patterns/ext4.hexpat`](patterns/ext4.hexpat) | Ext4 filesystem |
| EXFAT | | [`patterns/fs/exfat.hexpat`](patterns/fs/exfat.hexpat) | Extensible File Allocation Table (exFAT) |
| EXT4 | | [`patterns/fs/ext4.hexpat`](patterns/fs/ext4.hexpat) | Ext4 File System |
| FAS | | [`patterns/fas_oskasoftware.hexpat`](patterns/fas_oskasoftware.hexpat) [`patterns/fas_oskasoftware_old.hexpat`](patterns/fas_oskasoftware_old.hexpat) (Old versions of Oska DeskMate) | Oska Software DeskMates FAS (Frames and Sequences) file |
| FAT32 | | [`patterns/fs/fat32.hexpat`](patterns/fs/fat32.hexpat) | FAT32 File System |
| FBX | | [`patterns/fbx.hexpat`](patterns/fbx.hexpat) | Kaydara FBX Binary |
| FDT | | [`patterns/fdt.hexpat`](patterns/fdt.hexpat) | Flat Linux Device Tree blob |
| FFX | | [`patterns/ffx/*`](https://gitlab.com/EvelynTSMG/imhex-ffx-pats) | Various Final Fantasy X files |
| File System | `application/x-ima` | [`patterns/fs.hexpat`](patterns/fs.hexpat) | Drive File System |
| File System | `application/x-ima` | [`patterns/fs/pattern.hexpat`](patterns/fs/pattern.hexpat) | Drive File System |
| FLAC | `audio/flac` | [`patterns/flac.hexpat`](patterns/flac.hexpat) | Free Lossless Audio Codec, FLAC Audio Format |
| FLC/FLIC | | [`patterns/flc.hexpat`](patterns/flc.hexpat) | FLC/FLIC animation file |
| FLV | | [`patterns/flv.hexpat`](patterns/flv.hexpat) | FLv animation file |
| Flipper Zero Settings | | [`patterns/flipper_settings.hexpat`](patterns/flipper_settings.hexpat) | Flipper Zero Settings Files |
| GB | `application/x-gameboy-rom` | [`patterns/gb.hexpat`](patterns/gb.hexpat) | Game Boy ROM |
| GBA | `application/x-gameboy-advance-rom` | [`patterns/gba.hexpat`](patterns/gba.hexpat) | Game Boy Advance ROM header |
@@ -83,9 +92,11 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| Halo Tag || [`patterns/hinf_tag.hexpat`](patterns/hinf_tag.hexpat) | Halo Infinite Tag Files |
| Halo Module || [`patterns/hinf_module.hexpat`](patterns/hinf_module.hexpat) | Halo Infinite Module Archive Files |
| Halo HavokScript || [`patterns/hinf_luas.hexpat`](patterns/hinf_luas.hexpat) | Halo Infinite HavokScript 5.1 Bytecode |
| HPROF || [`patterns/hprof.hexpat`](patterns/hprof.hexpat) | Java HPROF Profiler Data Format |
| HSDT || [`patterns/hsdt.hexpat`](patterns/hsdt.hexpat) | HiSilicon device-tree table images |
| ICO | | [`patterns/ico.hexpat`](patterns/ico.hexpat) | Icon (.ico) or Cursor (.cur) files |
| ID3 | `audio/mpeg` | [`patterns/id3.hexpat`](patterns/id3.hexpat) | ID3 tags in MP3 files |
| IM*H || [`patterns/imah.hexpat`](patterns/imah.hexpat) | DJI Signed Firmware (IM*H) |
| Intel HEX | | [`patterns/intel_hex.hexpat`](patterns/intel_hex.hexpat) | [Intel hexadecimal object file format definition]("https://en.wikipedia.org/wiki/Intel_HEX") |
| IP | | [`patterns/ip.hexpat`](patterns/ip.hexpat) | Ethernet II Frames (IP Packets) |
| IPS | | [`patterns/ips.hexpat`](patterns/ips.hexpat) | IPS (International Patching System) files |
@@ -102,25 +113,34 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| Mach-O | `application/x-mach-binary` | [`patterns/macho.hexpat`](patterns/macho.hexpat) | Mach-O executable |
| MIDI | `audio/midi` | [`patterns/midi.hexpat`](patterns/midi.hexpat) | MIDI header, event fields provided |
| MiniDump | `application/x-dmp` | [`patterns/minidump.hexpat`](patterns/minidump.hexpat) | Windows MiniDump files |
| MO | | [`patterns/mo.hexpat`](patterns/mo.hexpat) | GNU Machine Object (MO) files containing translations for gettext |
| mp4 | `video/mp4` | [`patterns/mp4.hexpat`](patterns/mp4.hexpat) | MPEG-4 Part 14 digital multimedia container format |
| msgpack | `application/x-msgpack` | [`patterns/msgpack.hexpat`](patterns/msgpack.hexpat) | MessagePack binary serialization format |
| MSSCMP | | [`patterns/msscmp.hexpat`](patterns/msscmp.hexpat) | Miles Sound System Compressed Archive |
| NACP | | [`patterns/nacp.hexpat`](patterns/nacp.hexpat) | Nintendo Switch NACP files |
| NBT | | [`patterns/nbt.hexpat`](patterns/nbt.hexpat) | Minecraft NBT format |
| NDS | `application/x-nintendo-ds-rom` | [`patterns/nds.hexpat`](patterns/nds.hexpat) | DS Cartridge Header |
| NE | `application/x-ms-ne-executable` | [`patterns/ne.hexpat`](patterns/ne.hexpat) | NE header and Standard NE fields |
| nes | | [`patterns/nes.hexpat`](patterns/nes.hexpat) | .nes file format |
| NotepadCache | | [`patterns/notepad-cache.hexpat`](patterns/notepad-cache.hexpat) | Windows Notepad Cache |
| NotepadWindowState | | [`patterns/notepadwindowstate.hexpat`](patterns/notepadwindowstate.hexpat) | Windows 11 Notepad - Window State .bin file |
| NRO | | [`patterns/nro.hexpat`](patterns/nro.hexpat) | Nintendo Switch NRO files |
| NTAG | | [`patterns/ntag.hexpat`](patterns/ntag.hexpat) | NTAG213/NTAG215/NTAG216, NFC Forum Type 2 Tag compliant IC |
| NTFS | | [`patterns/fs/ntfs.hexpat`](patterns/fs/ntfs.hexpat) | NTFS (NT File System) |
| OGG | `audio/ogg` | [`patterns/ogg.hexpat`](patterns/ogg.hexpat) | OGG Audio format |
| ORP / ORS | | [`patterns/orp.hexpat`](patterns/orp.hexpat) | OpenRGB profile format |
| PACK | | [`patterns/roblox_pack.hexpat`](patterns/roblox_pack.hexpat) | Roblox shader archive format |
| PAK | | [`patterns/xgspak.hexpat`](patterns/xgspak.hexpat) | Exient XGS Engine Pak files |
| PCAP | `application/vnd.tcpdump.pcap` | [`patterns/pcap.hexpat`](patterns/pcap.hexpat) | pcap header and packets |
| PcapNG | `application/vnd.tcpdump.pcap` | [`patterns/pcapng.hexpat`](patterns/pcapng.hexpat) | pcapng header and packets |
| PCK | | [`patterns/pck.hexpat`](patterns/pck.hexpat) | Minecraft Legacy Console Edition .pck file |
| PCX | `application/x-pcx` | [`patterns/pcx.hexpat`](patterns/pcx.hexpat) | PCX Image format |
| PE | `application/x-dosexec` `application/x-msdownload` | [`patterns/pe.hexpat`](patterns/pe.hexpat) | PE header, COFF header, Standard COFF fields and Windows Specific fields |
| PEF | | [`patterns/pef.hexpat`](patterns/pef.hexpat) | Preffered Executable Format executable (for Mac OS 7.1.2 - Mac OS 10.4 / BeOS) |
| PEX | | [`patterns/pex.hexpat`](patterns/pex.hexpat) | Bethesda Papyrus executable compiled script file |
| PP | | [`patterns/selinuxpp.hexpat`](patterns/selinuxpp.pat) | SE Linux package |
| PFS0 | | [`patterns/pfs0.hexpat`](patterns/pfs0.hexpat) | Nintendo Switch PFS0 archive (NSP files) |
| PF | | [`patterns/pf.hexpat`](patterns/pf.hexpat) | Microsoft uncompressed prefetch files (.pf) |
| PIF | `image/pif` | [`patterns/pif.hexpat`](patterns/pif.hexpat) | PIF Image Format |
| PKM | | [`patterns/pkm.hexpat`](patterns/pkm.hexpat) | PKM texture format |
| PNG | `image/png` | [`patterns/png.hexpat`](patterns/png.hexpat) | PNG image files |
@@ -131,36 +151,50 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| PYC | `application/x-bytecode.python` | [`patterns/pyc.hexpat`](patterns/pyc.hexpat) | Python bytecode files |
| QBCL | | [`patterns/qbcl.hexpat`](patterns/qbcl.hexpat) | Qubicle voxel scene project file |
| QOI | `image/qoi` | [`patterns/qoi.hexpat`](patterns/qoi.hexpat) | QOI image files |
| Quake 3 engine demo | | [`patterns/q3demo.hexpat`](patterns/q3demo.hexpat) | Demos/replays of most Quake 3 engine games |
| quantized-mesh | | [`patterns/quantized-mesh.hexpat`](patterns/quantized-mesh.hexpat) | Cesium quantized-mesh terrain |
| RAR | `application/x-rar` | [`patterns/rar.hexpat`](patterns/rar.hexpat) | RAR archive file format |
| RAS | `image/x-sun-raster` | [`patterns/ras.hexpat`](patterns/ras.hexpat) | RAS image files |
| ReFS | | [`patterns/refs.hexpat`](patterns/refs.hexpat) | Microsoft Resilient File System |
| RCF 1.2 | | [`patterns/rcf_v1_2.hexpat`](patterns/rcf_v1_2.hexpat) | Radcore Cement Library 1.2 file header |
| ReFS | | [`patterns/refs.hexpat`](patterns/fs/refs.hexpat) | Microsoft Resilient File System |
| RGBDS | | [`patterns/rgbds.hexpat`](patterns/rgbds.hexpat) | [RGBDS](https://rgbds.gbdev.io) object file format |
| RPM | | [`patterns/rpm.hexpat`](patterns/rpm.hexpat) | [RPM](http://ftp.rpm.org/max-rpm/s1-rpm-file-format-rpm-file-format.html) package file format |
| SDB | | [`patterns/sdb.hexpat`](patterns/sdb.hexpat) | [Shim DataBase](https://learn.microsoft.com/en-us/windows/win32/devnotes/application-compatibility-database) file format |
| Shell Link | `application/x-ms-shortcut` | [`patterns/lnk.hexpat`](patterns/lnk.hexpat) | Windows Shell Link file format |
| shp | | [`patterns/shp.hexpat`](patterns/shp.hexpat) | ESRI shape file |
| SHR | | [`patterns/SHR.hexpat`](patterns/SHR.hexpat) | Apple IIgs Super Hi-Res (SHR) + PaintWorks Animation (ANI) |
| shx | | [`patterns/shx.hexpat`](patterns/shx.hexpat) | ESRI index file |
| smk | | [`patterns/smk.hexpat`](patterns/smk.hexpat) | Smacker video file |
| sup | | [`patterns/sup.hexpat`](patterns/sup.hexpat) | PGS Subtitle |
| SPIRV | | [`patterns/spirv.hexpat`](patterns/spirv.hexpat) | SPIR-V header and instructions |
| STDF | | [`patterns/stdfv4.hexpat`](patterns/stdfv4.hexpat) | Standard test data format for IC testers |
| STL | `model/stl` | [`patterns/stl.hexpat`](patterns/stl.hexpat) | STL 3D Model format |
| StuffItV5 | `application/x-stuffit` | [`patterns/sit5.hexpat`](patterns/sit5.hexpat) | StuffIt V5 archive |
| SQLite3 | `application/vnd.sqlite3` | [`patterns/sqlite3.hexpat`](patterns/sqlite3.hexpat) | SQLite3 Database |
| SWF | `application/x-shockwave-flash` |[`patterns/swf.hexpat`](patterns/swf.hexpat) | Shockwave Flash file format |
| TA | | [`patterns/optee_ta.hexpat`](patterns/optee_ta.hexpat) | OPTEE Trusted Application Executable |
| TAR | `application/x-tar` | [`patterns/tar.hexpat`](patterns/tar.hexpat) | Tar file format |
| TARC | | [`patterns/tarc.hexpat`](patterns/tarc.hexpat) | KEX Engine TARC file format |
| TES | | [`patterns/wintec_tes.hexpat`](patterns/wintec_tes.hexpat) | Wintec TES GPS log |
| Thumbcache | | [`patterns/thumbcache.hexpat`](patterns/thumbcache.hexpat) | Windows thumbcache_*.db |
| TIFF | `image/tiff` | [`patterns/tiff.hexpat`](patterns/tiff.hexpat) | Tag Image File Format |
| TGA | `image/tga` | [`patterns/tga.hexpat`](patterns/tga.hexpat) | Truevision TGA/TARGA image |
| TTF | `font/ttf`, `font/otf` | [`patterns/ttf.hexpat`](patterns/ttf.hexpat) | TrueType and OpenType font format |
| Ubiquiti | | [`patterns/ubiquiti.hexpat`](patterns/ubiquiti.hexpat) | Ubiquiti Firmware (update) image |
| UPK | | [`patterns/upk-ue3.hexpat`](patterns/upk-ue3.hexpat) | Unreal Engine 3 UPK file |
| UEFI | | [`patterns/uefi.hexpat`](patterns/uefi.hexpat)` | UEFI structs for parsing efivars |
| UEFI Boot Entry | | [`patterns/uefi_boot_entry.hexpat`](patterns/uefi_boot_entry.hexpat) | UEFI Boot Entry (Load option) |
| UEFI Variable Store | | [`patterns/uefi_fv_varstore.hexpat`](patterns/uefi_fv_varstore.hexpat) | UEFI Firmware Volume Variable Store |
| UF2 | | [`patterns/uf2.hexpat`](patterns/uf2.hexpat) | [USB Flashing Format](https://github.com/microsoft/uf2) |
| Valve VPK | | [`patterns/valve_vpk.hexpat`](valve_vpk.hexpat) | Valve Package File |
| VBMeta | | [`patterns/vbmeta.hexpat`](patterns/vbmeta.hexpat) | Android VBMeta image |
| VDF | | [`patterns/vdf.hexpat`](patterns/vdf.hexpat) | Binary Value Data Format (.vdf) files |
| VEADO | | [`patterns/veado.hexpat`](patterns/veado.hexpat) | veadotube mini avatar file |
| VGM | | [`patterns/vgm.hexpat`](patterns/vgm.hexpat) | VGM (Video Game Music) sound log |
| VHDX | | [`patterns/vhdx.hexpat`](patterns/vhdx.hexpat) | Microsoft Hyper-V Virtual Hard Disk format |
| VOX | | [`patterns/vox.hexpat`](patterns/vox.hexpat) | MagicaVoxel scene description format |
| WAV | `audio/x-wav` | [`patterns/wav.hexpat`](patterns/wav.hexpat) | RIFF header, WAVE header, PCM header |
| WAS | | [`patterns\was_oskasoftware.hexpat`](patterns\was_oskasoftware.hexpat) | Oska Software DeskMates WAS/WA3 (WAVE/MP3 Set) file
| WAS | | [`patterns/was_oskasoftware.hexpat`](patterns/was_oskasoftware.hexpat) | Oska Software DeskMates WAS/WA3 (WAVE/MP3 Set) file
| WAD | | [`patterns/wad.hexpat`](patterns/wad.hexpat) | DOOM WAD Archive |
| WebP | `image/webp` | [`patterns/webp.hexpat`](patterns/webp.hexpat) | Google WebP image |
| XBEH | `audio/x-xbox-executable` | [`patterns/xbeh.hexpat`](patterns/xbeh.hexpat) | Xbox executable |
@@ -168,9 +202,12 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| XGT | | [`patterns/xgt.hexpat`](patterns/xgstexture.hexpat) | Exient XGS Engine Texture |
| Xilinx BIT | | [`patterns/xilinx_bit.hexpat`](patterns/xilinx_bit.hexpat) | Xilinx FPGA Bitstreams |
| Xilinx Bootgen | `application/x-xilinx-boot-zynqmp` | [`patterns/xilinx_bootgen.hexpat`](patterns/xilinx_bootgen.hexpat) | Xilinx ZynqMP Boot Images |
| ZIM | | [`patterns/zim.hexpat`](patterns/zim.hexpat) | [ZIM](https://openzim.org) file format |
| ZIP | `application/zip` | [`patterns/zip.hexpat`](patterns/zip.hexpat) | End of Central Directory Header, Central Directory File Headers |
| ZLIB | `application/zlib` | [`patterns/zlib.hexpat`](patterns/zlib.hexpat) | ZLIB compressed data format |
| ZSTD | `application/zstd` | [`patterns/zstd.hexpat`](patterns/zstd.hexpat) | Zstandard compressed data format |
| MOD | `3d-model/mod` | [`patterns/DMC3HD-Mod.hexpat`](patterns/dmc3_hd_mod.hexpat) | 3D Model files used in Devil May Cry 3 HD Collection |
| CBM BASIC | | [`commodore_basic.hexpat`](patterns/commodore_basic.hexpat) | Commodore BASIC |
### Scripts
@@ -266,6 +303,8 @@ Everything will immediately show up in ImHex's Content Store and gets bundled wi
| Catppuccin Frappe | [`themes/catppuccin-frappe.json`](themes/catppuccin-frappe.json) | Catppuccin Frappe Flavor (Dark Theme) |
| Catppuccin Macchiato | [`themes/catppuccin-macchiato.json`](themes/catppuccin-macchiato.json) | Catppuccin Macchiato Flavor (Dark Theme) |
| Catppuccin Mocha | [`themes/catppuccin-mocha.json`](themes/catppuccin-mocha.json) | Catppuccin Mocha Flavor (Dark Theme) |
| Theme Lion | [`themes/theme_lion.json`](themes/theme_lion.json) | Semantic CLion inspired theme (Dark Theme) |
| Retina Dark | [`themes/retina_dark.json`](themes/retina_dark.json) | Semantic theme based on Dark Theme |
### Disassemblers

View File

@@ -49,8 +49,8 @@
"format": "#{I}"
},
{
"mask": "areturn",
"mnemonic": "1011'0000",
"mask": "1011'0000",
"mnemonic": "areturn",
"format": ""
},
{
@@ -84,8 +84,8 @@
"format": ""
},
{
"mask": "athrow",
"mnemonic": "1011'1111",
"mask": "1011'1111",
"mnemonic": "athrow",
"format": ""
},
{
@@ -1042,4 +1042,4 @@
"format": ""
}
]
}
}

View File

@@ -25,7 +25,7 @@ namespace auto hex::dec {
@param pattern The pattern whose bytes should be decompressed
@param section The section to decompress the data into
@param window_size The window size passed to zlib
@return true if successful, false otherwise
@return A value representing either the number of bytes decompressed or an error code from zlib
*/
fn zlib_decompress(ref auto pattern, std::mem::Section section, u64 window_size = 0) {
return builtin::hex::dec::zlib_decompress(pattern, section, window_size);
@@ -71,5 +71,5 @@ namespace auto hex::dec {
fn lz4_decompress(ref auto pattern, std::mem::Section section, bool frame = true) {
return builtin::hex::dec::lz4_decompress(pattern, section, frame);
};
}

View File

@@ -13,7 +13,7 @@ namespace auto hex::type {
@tparam Size size of the string
*/
struct Json<auto Size> {
char __data[Size] [[hidden]];
char __data[Size] [[hidden, no_unique_address]];
builtin::hex::dec::Json<__data> json [[merge]];
};
@@ -22,7 +22,7 @@ namespace auto hex::type {
@tparam Size size of the data
*/
struct Bson<auto Size> {
u8 __data[Size] [[hidden]];
u8 __data[Size] [[hidden, no_unique_address]];
builtin::hex::dec::Bson<__data> bson [[merge]];
};
@@ -31,7 +31,7 @@ namespace auto hex::type {
@tparam Size size of the data
*/
struct Cbor<auto Size> {
u8 __data[Size] [[hidden]];
u8 __data[Size] [[hidden, no_unique_address]];
builtin::hex::dec::Cbor<__data> cbor [[merge]];
};
@@ -40,7 +40,7 @@ namespace auto hex::type {
@tparam Size size of the data
*/
struct Bjdata<auto Size> {
u8 __data[Size] [[hidden]];
u8 __data[Size] [[hidden, no_unique_address]];
builtin::hex::dec::Bjdata<__data> bjdata [[merge]];
};
@@ -49,7 +49,7 @@ namespace auto hex::type {
@tparam Size size of the data
*/
struct Msgpack<auto Size> {
u8 __data[Size] [[hidden]];
u8 __data[Size] [[hidden, no_unique_address]];
builtin::hex::dec::Msgpack<__data> msgpack [[merge]];
};
@@ -58,7 +58,7 @@ namespace auto hex::type {
@tparam Size size of the data
*/
struct Ubjson<auto Size> {
u8 __data[Size] [[hidden]];
u8 __data[Size] [[hidden, no_unique_address]];
builtin::hex::dec::Ubjson<__data> ubjson [[merge]];
};

View File

@@ -6,6 +6,22 @@ import std.core;
/*!
The array library contains a helper type to make it easier to create multi-dimensional arrays
and pass arrays to functions as parameters.
## Multi-dimensional arrays
The following example shows how to use multi-dimensional arrays with structs.
```rust
import std.array;
struct File {
u8 width, height;
std::Array<std::Array<u8, parent.width>, height> cells;
};
File file @ 0x00;
```
*/
namespace auto std {
@@ -27,7 +43,7 @@ namespace auto std {
struct ByteSizedArray<T, auto NumBytes> {
u64 startAddress = $;
T array[while($ - startAddress < NumBytes)] [[inline]];
std::assert($ - startAddress == NumBytes, "Not enough bytes available to fit a whole number of types");
} [[format("std::impl::format_array")]];

View File

@@ -85,7 +85,7 @@ namespace auto std::ctype {
@return True if `c` is part of this range, false otherwise
*/
fn isprint(char c) {
return c >= '0' && c <= '~';
return c >= ' ' && c <= '~';
};
/**

View File

@@ -6,27 +6,12 @@
namespace auto std::mem {
namespace impl {
struct MagicSearchImpl<auto Magic, T> {
s128 address = builtin::std::mem::find_string_in_range(0, $, builtin::std::mem::size(), Magic);
if (address < 0)
break;
$ = address;
try {
T data [[inline]];
} catch {
T data;
}
};
}
/**
A Handle for a custom Section
*/
using Section = u128;
const u64 SectionMain = 0;
const u64 SectionCurrent = 0xFFFF'FFFF'FFFF'FFFF;
/**
The endianness of a value
@@ -72,16 +57,16 @@ namespace auto std::mem {
Gets the base address of the data
@return The base address of the memory
*/
fn base_address() {
return builtin::std::mem::base_address();
fn base_address(u64 section = SectionCurrent) {
return builtin::std::mem::base_address(section);
};
/**
Gets the size of the data
@return The size of the memory
*/
fn size() {
return builtin::std::mem::size();
fn size(u64 section = SectionCurrent) {
return builtin::std::mem::size(section);
};
/**
@@ -91,8 +76,8 @@ namespace auto std::mem {
@return The address of the sequence
*/
fn find_sequence(u128 occurrence_index, auto ... bytes) {
const u128 address = builtin::std::mem::base_address();
return builtin::std::mem::find_sequence_in_range(occurrence_index, address, address + builtin::std::mem::size(), bytes);
const u128 address = std::mem::base_address();
return builtin::std::mem::find_sequence_in_range(occurrence_index, address, address + std::mem::size(), bytes);
};
/**
@@ -115,8 +100,8 @@ namespace auto std::mem {
@return The address of the sequence
*/
fn find_string(u128 occurrence_index, str string) {
const u128 address = builtin::std::mem::base_address();
return builtin::std::mem::find_string_in_range(occurrence_index, address, address + builtin::std::mem::size(), string);
const u128 address = std::mem::base_address();
return builtin::std::mem::find_string_in_range(occurrence_index, address, address + std::mem::size(), string);
};
/**
@@ -138,8 +123,8 @@ namespace auto std::mem {
@param [endian] The endianness of the value to read. Defaults to native
@return The value read
*/
fn read_unsigned(u128 address, u8 size, Endian endian = Endian::Native) {
return builtin::std::mem::read_unsigned(address, size, u32(endian));
fn read_unsigned(u128 address, u8 size, Endian endian = Endian::Native, Section section = SectionCurrent) {
return builtin::std::mem::read_unsigned(address, size, u32(endian), section);
};
/**
@@ -149,8 +134,8 @@ namespace auto std::mem {
@param [endian] The endianness of the value to read. Defaults to native
@return The value read
*/
fn read_signed(u128 address, u8 size, Endian endian = Endian::Native) {
return builtin::std::mem::read_signed(address, size, u32(endian));
fn read_signed(u128 address, u8 size, Endian endian = Endian::Native, Section section = SectionCurrent) {
return builtin::std::mem::read_signed(address, size, u32(endian), section);
};
/**
@@ -159,8 +144,8 @@ namespace auto std::mem {
@param size The size of the value to read
@return The value read
*/
fn read_string(u128 address, u128 size) {
return builtin::std::mem::read_string(address, size);
fn read_string(u128 address, u128 size, Section section = SectionCurrent) {
return builtin::std::mem::read_string(address, size, section);
};
/**
@@ -242,16 +227,6 @@ namespace auto std::mem {
return builtin::std::mem::current_bit_offset();
};
/**
Searches for a sequence of bytes and places the given type at that address
@tparam Magic The magic sequence to search for
@tparam T The type to place at the address
*/
struct MagicSearch<auto Magic, T> {
std::mem::impl::MagicSearchImpl<Magic, T> impl[while(!std::mem::eof())] [[inline]];
};
/**
Reinterprets a value as a different one
@tparam From The type to reinterpret from
@@ -285,6 +260,28 @@ namespace auto std::mem {
return "";
};
struct MagicSearchImpl<auto Magic, T> {
s128 address = builtin::std::mem::find_string_in_range(0, $, std::mem::size(), Magic);
if (address < 0)
break;
$ = address;
try {
T data [[inline]];
} catch {
T data;
}
};
}
/**
Searches for a sequence of bytes and places the given type at that address
@tparam Magic The magic sequence to search for
@tparam T The type to place at the address
*/
struct MagicSearch<auto Magic, T> {
std::mem::impl::MagicSearchImpl<Magic, T> impl[while(!std::mem::eof())] [[inline]];
};
}

View File

@@ -47,9 +47,9 @@ namespace auto std::ptr {
Example:
A struct field called `p_myInfo` which is a nullable 64-bit pointer to an
element of type `MyInfoTy` would be written as:
```
```rust
struct MyStruct {
std::ptr::NullablePtr<MyInfoTy, u64> p_myInfo;
std::ptr::NullablePtr<MyInfoTy, u64> p_myInfo;
}
```
*/

View File

@@ -21,7 +21,7 @@ namespace auto type {
g : G;
b : B;
if (A > 0) a : A;
} [[sealed, format("type::impl::format_color"), color(std::format("{0:02X}{1:02X}{2:02X}FF", r, g, b))]];
} [[sealed, format("type::impl::format_color"), color(std::format("{0:02X}{1:02X}{2:02X}", r, g, b))]];
/**
Type representing a generic RGB color with a variable number of bits for each color
@@ -77,4 +77,4 @@ namespace auto type {
}
}
}

View File

@@ -11,7 +11,7 @@ namespace auto type
{
/**
Escapes all bytes in a string to only contain printable characters. All non-printable bytes will be transformed to sequences in the form form \xFF
Escapes all bytes in a string to only contain printable characters. All non-printable bytes will be transformed to sequences in the \xFF form
@param value Byte array to escape
@return Escaped string
*/

View File

@@ -1,9 +1,11 @@
#pragma description 7z Archive
#pragma MIME application/x-7z-compressed
#pragma magic [ 37 7A BC AF 27 1C ] @ 0x00
import std.io;
import std.mem;
import std.math;
import type.magic;
enum Type:u8{
@@ -11,13 +13,9 @@ enum Type:u8{
sizeStartHeader = 0x20, // Size of start Header
};
enum TypeB:u48{
sevenZipSignature = 0x1C27AFBC7A37, // Defining 7z signature
};
struct StartHeader {
// File signature
u48 signature [[color("FF0000")] ];
type::Magic<"7z\xBC\xAF\x27\x1C"> signature [[color("FF0000")] ];
// Version format
u16 formatVersion [[color("00FF00")]];
// CRC start header
@@ -62,13 +60,6 @@ struct EndHeader {
EndHeader endheader @ Type::sizeStartHeader;
// Check 7z type
if(startheader.signature == TypeB::sevenZipSignature){
std::print("It is a 7z File Type");
}else{
std::print("The file is not 7z type");
}
std::print("Format Version {} ",startheader.formatVersion);
// Version verification

189
patterns/SHR.hexpat Normal file
View File

@@ -0,0 +1,189 @@
/*!
Apple IIgs Super Hi-Res (SHR) + PaintWorks Animation (ANI) — ImHex pattern
Supports:
• PIC $C1/$0000 — 32 KB uncompressed SHR screen image
• PIC $C1/$0002 — 3200-color (“Brooks”) per-scanline palettes
• ANI $C2/$0000 — PaintWorks animation:
0x0000..0x7FFF : base SHR $C1/$0000 image
0x8000.. : animation header + chunks
PaintWorks animation structure (per reversed docs):
- Base image: uncompressed SHR ($C1/$0000)
- +0x8000 u32: total animation data length after header (file_len - 0x8008)
- +0x8004 u16: global frame delay in VBLs
- +0x8006 u16: flag/? (commonly 0x00C0 or 0x00C1)
- +0x8008 ...: one or more animation chunks:
chunk:
u32 chunk_len (includes this length field)
repeated { u16 offset; u16 value; } pairs
offset==0x0000 marks End-of-Frame (value is ignored)
- Offsets may target any byte in the 32 KB SHR space (pixels, SCBs, palettes).
This enables palette-cycling and SCB effects.
References:
- CiderPress2 PaintWorks Animation notes (file structure, fields, semantics).
*/
import std.core;
import std.sys;
import std.math;
#pragma endian little
#include <std/mem.pat>
#pragma description Apple IIgs Super Hi-Res (SHR) + PaintWorks Animation (ANI)
#pragma author hasseily
// ------------------------------ Constants ------------------------------
const u32 SHR_ROWS = 200;
const u32 SHR_BYTES_PER_ROW = 160;
const u32 SHR_PIXEL_DATA_SIZE = SHR_ROWS * SHR_BYTES_PER_ROW; // 32000 (0x7D00)
const u32 PIC0000_FILE_SIZE = 0x8000; // 32768
const u32 PIC0002_FILE_SIZE = 0x9600; // 38400 (32000 + 200*32)
const u32 PIC0000_OFF_PIXELS = 0x0000;
const u32 PIC0000_OFF_SCB = 0x7D00;
const u32 PIC0000_OFF_RESERVED = 0x7DC8;
const u32 PIC0000_OFF_PALETTES = 0x7E00;
const u32 PALETTE_COUNT = 16;
const u32 PALETTE_COLORS = 16;
const u32 ANI_BASE_SHR_SIZE = 0x8000; // First 32 KB is a $C1/$0000 image
const u32 ANI_HDR_OFF = 0x8000; // Animation header starts here
const u32 ANI_MIN_TOTAL_SIZE = 0x8008; // base + header (no chunks)
// ------------------------------ Types: SHR core ------------------------------
struct Row160 { u8 data[SHR_BYTES_PER_ROW]; };
// Scanline Control Byte
bitfield ShrSCB {
palette : 4; // 0..15
reserved : 1;
color_fill : 1;
interrupt : 1;
mode_640 : 1; // 0=320, 1=640
};
// helper: expand a 4-bit channel to 8-bit (0x0..0xF -> 0x00..0xFF)
fn expand4(u8 v) { return (v << 4) | v; };
bitfield Colors_gb {
blue : 4; // Blue (B3..B0)
green : 4; // Green (G3..G0)
};
bitfield Colors_r0 {
red : 4; // Red (R3..R0)
unused : 4; // Unused / reserved
};
// RGB444 stored as 0RGB
struct Rgb444_0RGB {
Colors_gb gb;
Colors_r0 r0;
} [[color(std::format("{:02X}{:02X}{:02X}", expand4(r0.red), expand4(gb.green), expand4(gb.blue)))]];
struct Palette16 { Rgb444_0RGB color[PALETTE_COLORS]; };
// $C1/$0000 raw 32 KB screen dump
struct SHR_PIC0000 {
Row160 pixels[SHR_ROWS] @ PIC0000_OFF_PIXELS;
ShrSCB scb[SHR_ROWS] @ PIC0000_OFF_SCB;
u8 reserved[56] @ PIC0000_OFF_RESERVED;
Palette16 palettes[PALETTE_COUNT] @ PIC0000_OFF_PALETTES;
};
// “Brooks” 3200-color: pixels + 200 per-line palettes (no SCBs)
struct BrooksLinePalette { Rgb444_0RGB color[PALETTE_COLORS]; };
struct SHR_PIC0002 {
Row160 pixels[SHR_ROWS] @ 0x0000;
BrooksLinePalette line_palettes[SHR_ROWS] @ SHR_PIXEL_DATA_SIZE; // 0x7D00
};
// ------------------------------ Types: PaintWorks ANI ($C2/$0000) ------------------------------
/* Each operation modifies 1 word at an absolute offset in the 32 KB SHR area.
End-of-frame marker: offset == 0x0000 (value is ignored). */
struct AniOp {
u16 offset [[color("0000AA")]]; // 0x0000..0x7FFE valid; 0x0000 = End-of-Frame
u16 value [[color("00AAAA")]]; // word to store at [offset]
// For convenience in the sidebar:
bool is_eof = (offset == 0x0000);
};
// A contiguous animation chunk: length + packed AniOp pairs.
// Most files have exactly one chunk that spans all frames.
struct AniChunk {
u32 chunk_len; // includes this field
// ops_count = (chunk_len - 4)/4, unless chunk_len == 4
// in which case: (__file_size - 0x8000 - 12)/4
u64 ops_count64 =
(chunk_len == 4)
? ( (__file_size > (0x8000 + 12)) ? ((__file_size - 0x8000 - 12) / 4) : 0 )
: ( (chunk_len >= 4) ? (u64(chunk_len - 4) / 4) : 0 );
u32 ops_count = u32(ops_count64);
// ops start immediately after chunk_len (offset +4)
AniOp ops[ops_count];
};
// Header located at 0x8000 after the base 32 KB image
struct AniHeader {
u32 anim_data_len [[color("660000")]]; // total bytes of animation data after header
u16 frame_delay_vbl [[color("CC0000")]]; // global per-frame delay in VBLs (NTSC/PAL differ)
u16 flag_unknown; // usually 0x00C0 or 0x00C1
};
// Full PaintWorks animation container
struct ANI_PaintWorks {
// Base frame: a normal uncompressed SHR image
SHR_PIC0000 base @ 0x0000;
// Global animation header
AniHeader hdr @ ANI_HDR_OFF;
// One or more chunks, typically exactly one:
AniChunk chunks[ std::math::min(
u32(16), // cap to keep ImHex happy in pathological cases
u32((__file_size - (ANI_HDR_OFF + sizeof(AniHeader))) > 3 ?
1 + u32((__file_size - (ANI_HDR_OFF + sizeof(AniHeader))) / 0x10000000) :
1)
) ] @ (ANI_HDR_OFF + sizeof(AniHeader));
// Helpful computed values for inspection:
u64 file_len = __file_size;
u64 expected_anim_end = ANI_HDR_OFF + sizeof(AniHeader) + u64(hdr.anim_data_len);
};
// ------------------------------ Dispatcher ------------------------------
u64 __file_size = std::mem::size();
if (__file_size == PIC0000_FILE_SIZE) {
// Plain SHR dump
SHR_PIC0000 pic0000 @ 0x0000;
} else if (__file_size == PIC0002_FILE_SIZE) {
// Brooks 3200-color
SHR_PIC0002 pic0002 @ 0x0000;
} else if (__file_size >= ANI_MIN_TOTAL_SIZE) {
// Heuristic: treat as PaintWorks ANI if theres room for base+header.
// (Many PW ANI files use ProDOS type $C2/$0000.)
ANI_PaintWorks ani @ 0x0000;
} else if (__file_size >= SHR_PIXEL_DATA_SIZE) {
// Fallback: show pixels only for odd dumps
Row160 pixels_only[SHR_ROWS] @ 0x0000;
}
ANI_PaintWorks ani_paintworks_at_0 @ 0;

176
patterns/adtfdat.hexpat Normal file
View File

@@ -0,0 +1,176 @@
#pragma description ADTFDAT File
#pragma magic [ 49 46 48 44 ] @ 0x00
#pragma endian little
import std.io;
import std.mem;
import type.time;
enum FileVersion : u32 {
with_history_end_offset = 0x00000301,
v500 = 0x00000500
};
bitfield ChunkFlags {
key : 1;
info : 1;
marker : 1;
type : 1;
trigger : 1;
reserved: 11;
};
struct Extension
{
char identifier[384];
u16 stream_id;
u8 reserved1[2];
u32 user_id;
u32 type_id;
u32 version_id;
u64 data_pos;
u64 data_size;
u8 reserved[96];
u8 payload[data_size] @ data_pos;
};
struct ChunkHeader {
u64 time_stamp [[format("format_timestamp_with_offset")]];;
u32 ref_master_table_index;
u32 offset_to_last;
u32 size;
u16 stream_id;
ChunkFlags flags;
u64 stream_index;
};
struct SampleInfo {
u8 memory_layout_version;
u32 size;
u8 payload[size];
};
struct Sample {
s64 timestamp [[format("format_timestamp")]];;
s32 flags;
u64 buffer_size;
u8 payload[buffer_size];
if (flags & 0x100) {
SampleInfo sample_info;
}
if (flags & 0x200) {
u32 substream_id;
} else {
u32 substream_id = 0 [[export]];
}
};
struct Chunk {
auto start_address = $;
ChunkHeader header;
if (header.size < 32 || start_address + header.size > std::mem::size()) {
std::warning(std::format("Invalid header size {} in chunk {} at offset 0x{:X}, last valid chunk at 0x{:X}.", header.size, chunk_index, start_address, last_valid_chunk_offset));
break;
}
last_valid_chunk_offset = start_address;
u8 payload[header.size - 32];
if (!skip_valid_chunks) {
if (!header.flags.info && !header.flags.marker && !header.flags.type && !header.flags.trigger) {
try {
Sample sample @ addressof(payload);
} catch {
std::warning(std::format("Invalid sample payload at chunk {} at 0x{:X}", chunk_index, start_address));
}
}
}
chunk_index += 1;
// Padding with 0xEE to the next chunk header
std::mem::AlignTo<16>;
if (skip_valid_chunks) {
continue;
}
};
struct Header {
u32 file_id;
u32 version_id;
u32 flags;
u32 extension_count;
u64 extension_offset;
u64 data_offset;
u64 data_size;
u64 chunk_count;
u64 max_chunk_size;
u64 duration;
type::time64_t filetime;
u8 header_byte_order;
u64 time_offset [[format("format_timestamp")]];
u8 patch_number;
u64 first_chunk_offset;
u64 continuous_offset;
u64 ring_buffer_end_offset;
u8 reserved[30];
char description[1912];
};
struct IndexedFile {
Header header;
Extension extensions[header.extension_count] @ header.extension_offset;
if (header.version_id >= FileVersion::with_history_end_offset &&
header.continuous_offset != header.data_offset) {
try {
Chunk ring_front[while($ < header.continuous_offset)] @ header.first_chunk_offset;
Chunk ring_back[while($ < header.ring_buffer_end_offset)] @ header.data_offset;
Chunk continueous[header.chunk_count - chunk_index] @ header.continuous_offset;
} catch {
std::warning("Too many chunks. Performing check only.");
skip_valid_chunks = true;
chunk_index = 0;
Chunk ring_front[while($ < header.continuous_offset)] @ header.first_chunk_offset;
Chunk ring_back[while($ < header.ring_buffer_end_offset)] @ header.data_offset;
Chunk continueous[while(chunk_index < header.chunk_count)] @ header.continuous_offset;
}
} else {
try {
Chunk chunks[header.chunk_count] @ header.data_offset;
} catch {
std::warning("Too many chunks. Performing check only.");
skip_valid_chunks = true;
chunk_index = 0;
Chunk chunks[while(chunk_index < header.chunk_count)] @ header.data_offset;
}
}
};
fn format_timestamp(u64 data) {
double seconds = 0;
if (file.header.version_id < FileVersion::v500) {
// microseconds
seconds = data / double(1000000);
} else {
// nanoseconds
seconds = data / double(1000000000);
}
std::time::Time time64 = std::time::to_utc(seconds);
return std::time::format(time64);
};
fn format_timestamp_with_offset(u64 data) {
return format_timestamp(data + file.header.time_offset);
};
bool skip_valid_chunks = false;
u64 last_valid_chunk_offset = 0;
u64 chunk_index = 0;
IndexedFile file @ 0x00;

View File

@@ -36,7 +36,12 @@ struct RiffChunk {
u32 size;
if (signature == "RIFF" || signature == "LIST") {
char type[4];
RiffChunk chunks[while($ - addressof(type) < size)];
// Compatible with error size defined in ani files
u32 remain = sizeof($) - addressof(type);
u32 marked = size > remain ? remain : size;
RiffChunk chunks[while($ - addressof(type) < marked)];
} else if (signature[0] == 'I' && parent.type == "INFO") {
char info[size];
} else if (signature == "anih") {

View File

@@ -0,0 +1,50 @@
#pragma author Lexi Mayfield
#pragma description AppleSingle/AppleDouble file format
#pragma endian big
#pragma magic [00 05 16 0?] @ 0x00
import type.magic;
enum EntryID : u32 {
DataFork = 1,
ResourceFork = 2,
RealName = 3,
Comment = 4,
IconBW = 5,
IconColor = 6,
FileDatesInfo = 8,
FinderInfo = 9,
MacFileInfo = 10,
ProDOSFileInfo = 11,
MSDOSFileInfo = 12,
ShortName = 13,
AFPFileInfo = 14,
DirectoryID = 15,
};
struct Entry {
EntryID entryID;
u32 offset;
u32 length;
u8 data[length] @ offset [[sealed]];
};
enum FileType : u32 {
AppleSingle = 0x00051600,
AppleDouble = 0x00051607,
};
enum Version : u32 {
V1 = 0x00010000,
V2 = 0x00020000,
};
struct AppleSingleDouble {
FileType fileType;
Version version;
char homeFileSystem[16];
u16 numEntries;
Entry entryDescs[numEntries];
};
AppleSingleDouble appleSingleDouble @ 0x00;

View File

@@ -38,10 +38,15 @@ namespace bencode {
ASCIIDecimal length;
char separator [[hidden]];
char value[length];
} [[sealed, format("bencode::format_string"), transform("bencode::format_string")]];
} [[sealed, format("bencode::format_string")]];
fn format_string(String string) {
return string.value;
for (u64 i = 0, i < string.length, i = i + 1) {
if (!std::ctype::isprint(string.value[i])) {
return "Contains non-printable characters";
}
}
return std::format("\"{}\"", string.value);
};
using Bencode;
@@ -57,6 +62,10 @@ namespace bencode {
if (type == Type::Dictionary) {
DictionaryEntry entry[while(std::mem::read_unsigned($, 1) != 'e')];
char end;
} else if (type == Type::List) {
Value entry[while(std::mem::read_unsigned($, 1) != 'e')];
char end;
} else if (type == Type::Integer) {
ASCIIDecimal value;
char end;

105
patterns/chd.hexpat Normal file
View File

@@ -0,0 +1,105 @@
#pragma author Lexi Mayfield
#pragma description MAME Compressed Hunks of Data
#pragma endian big
fn CHD_MAKE_TAG(char a, char b, char c, char d) {
return (u32(u8(a)) << 24) |
u32(u8((b)) << 16) |
u32(u8((c)) << 8) |
u32(u8(d));
};
enum CHDv5_CODEC : u32 {
NONE = 0,
ZLIB = CHD_MAKE_TAG('z','l','i','b'),
ZSTD = CHD_MAKE_TAG('z','s','t','d'),
LZMA = CHD_MAKE_TAG('l','z','m','a'),
HUFFMAN = CHD_MAKE_TAG('h','u','f','f'),
FLAC = CHD_MAKE_TAG('f','l','a','c'),
CD_ZLIB = CHD_MAKE_TAG('c','d','z','l'),
CD_ZSTD = CHD_MAKE_TAG('c','d','z','s'),
CD_LZMA = CHD_MAKE_TAG('c','d','l','z'),
CD_FLAC = CHD_MAKE_TAG('c','d','f','l'),
AVHUFF = CHD_MAKE_TAG('a','v','h','u'),
};
enum CHDv5_METADATA_TAG : u32 {
CHDMETATAG_WILDCARD = 0,
HARD_DISK_METADATA_TAG = CHD_MAKE_TAG('G','D','D','D'),
HARD_DISK_IDENT_METADATA_TAG = CHD_MAKE_TAG('I','D','N','T'),
HARD_DISK_KEY_METADATA_TAG = CHD_MAKE_TAG('K','E','Y',' '),
PCMCIA_CIS_METADATA_TAG = CHD_MAKE_TAG('C','I','S',' '),
CDROM_OLD_METADATA_TAG = CHD_MAKE_TAG('C','H','C','D'),
CDROM_TRACK_METADATA_TAG = CHD_MAKE_TAG('C','H','T','R'),
CDROM_TRACK_METADATA2_TAG = CHD_MAKE_TAG('C','H','T','2'),
GDROM_OLD_METADATA_TAG = CHD_MAKE_TAG('C','H','G','T'),
GDROM_TRACK_METADATA_TAG = CHD_MAKE_TAG('C','H','G','D'),
DVD_METADATA_TAG = CHD_MAKE_TAG('D','V','D',' '),
AV_METADATA_TAG = CHD_MAKE_TAG('A','V','A','V'),
AV_LD_METADATA_TAG = CHD_MAKE_TAG('A','V','L','D'),
};
struct CHDv5UncompressedMap {
u32 offset;
};
struct CHDv5CompressedMapEntry {
u8 compression;
u24 complength;
u48 offset;
u16 crc;
};
struct CHDv5CompressedMap {
u32 length;
u48 datastart;
u16 crc;
u8 lengthbits;
u8 hunkbits;
u8 parentunitbits;
u8 reserved;
};
struct CHDv5MetadataEntry {
CHDv5_METADATA_TAG metatag;
u8 flags;
u24 length;
u64 next;
char entry[length];
if (next != 0) {
CHDv5MetadataEntry nextMeta @ next;
}
};
struct CHDv5 {
CHDv5_CODEC compressors[4];
u64 logicalbytes;
u64 mapoffset;
u64 metaoffset;
u32 hunkbytes;
u32 unitbytes;
u8 rawsha1[20];
u8 sha1[20];
u8 parentsha1[20];
if (compressors[0] == CHDv5_CODEC::NONE) {
CHDv5UncompressedMap map @ mapoffset;
} else {
CHDv5CompressedMap map @ mapoffset;
}
CHDv5MetadataEntry meta @ metaoffset;
};
struct CHD {
char tag[8];
u32 length;
u32 version;
if (version == 5) {
CHDv5 chd;
}
};
CHD chd @ 0x00;

View File

@@ -0,0 +1,158 @@
#pragma description Commodore BASIC
#pragma author Stephen Hewitt
import std.io;
import std.mem;
bool in_quotes = false;
le u16 LoadAddress @0 [[color("ff0000")]];
fn formatll(u16 offset) {
if (offset&0xff00 == 0)
return "No next line";
u16 fo = offset-LoadAddress+2;
return std::format("Next line: ${:04X} (offset ${:04X})", offset, fo);
};
enum Token : u8 {
END = 0x80,
FOR = 0x81,
NEXT = 0x82,
DATA = 0x83,
INPUT_ = 0x84,
INPUT = 0x85,
DIM = 0x86,
READ = 0x87,
LET = 0x88,
GOTO = 0x89,
RUN = 0x8a,
IF = 0x8b,
RESTORE = 0x8c,
GOSUB = 0x8d,
RETURN = 0x8e,
REM = 0x8f,
STOP = 0x90,
ON = 0x91,
WAIT = 0x92,
LOAD = 0x93,
SAVE = 0x94,
VERIFY = 0x95,
DEF = 0x96,
POKE = 0x97,
PRINT_ = 0x98,
PRINT = 0x99,
CONT = 0x9a,
LIST = 0x9b,
CLR = 0x9c,
CMD = 0x9d,
SYS = 0x9e,
OPEN = 0x9f,
CLOSE = 0xa0,
GET = 0xa1,
NEW = 0xa2,
TAB_ = 0xa3,
TO = 0xa4,
FN = 0xa5,
SPC_ = 0xa6,
THEN = 0xa7,
NOT = 0xa8,
STEP = 0xa9,
PLUS_ = 0xaa,
MINUS_ = 0xab,
TIMES_ = 0xac,
DIVIDE_ = 0xad,
POW_ = 0xae,
AND = 0xaf,
OR = 0xb0,
GT_ = 0xb1,
EQ_ = 0xb2,
LT_ = 0xb3,
SGN = 0xb4,
INT = 0xb5,
ABS = 0xb6,
USR = 0xb7,
FRE = 0xb7,
POS = 0xb9,
SQR = 0xba,
RND = 0xbb,
LOG = 0xbc,
EXP = 0xbd,
COS = 0xbe,
SIN = 0xbf,
TAN = 0xc0,
ATN = 0xc1,
PEEK = 0xc2,
LEN = 0xc3,
STR_ = 0xc4,
VAL = 0xc5,
ASC = 0xc6,
CHR_ = 0xc7,
LEFT_ = 0xc8,
RIGHT_ = 0xc9,
MID_ = 0xca,
PI_ = 0xff
} [[format("formate")]];
// Can't seem to put attributes on enum members. Hack around it.
fn formate(Token t) {
match (t) {
(Token::INPUT_): return "INPUT#"; // $84
(Token::PRINT_): return "PRINT#"; // $98
(Token::TAB_): return "TAB("; // $a3
(Token::SPC_): return "SPC("; // $a6
(Token::PLUS_): return "+"; // $aa
(Token::MINUS_): return "-"; // $ab
(Token::TIMES_): return "*"; // $ac
(Token::DIVIDE_): return "/"; // $ad
//(Token::POW_): return "↑"; // $ae
(Token::GT_): return ">"; // $b1
(Token::EQ_): return "="; // $b2
(Token::LT_): return "<"; // $b3
(Token::STR_): return "STR$"; // $c4
(Token::CHR_): return "CHR$"; // $c7
(Token::LEFT_): return "LEFT$"; // $c8
(Token::RIGHT_): return "RIGHT$"; // $c9
(Token::MID_): return "MID$"; // $ca
//(Token::PI_): return "π"; // $ff
}
return t;
};
fn NotZero() {
u8 b = std::mem::read_unsigned($, 1);
return b!=0;
};
fn IsToken() {
u8 b = std::mem::read_unsigned($, 1);
return b&0x80!=0;
};
fn IsPETSCII() {
u8 b = std::mem::read_unsigned($, 1);
if (b == '"')
in_quotes = !in_quotes;
return (b!=0) && (in_quotes || (b&0x80)==0);
};
struct LineSegment
{
Token tokens[while(IsToken())] [[color("a040a0")]];
char petscii[while(IsPETSCII())] [[color("a0a0a0")]];
};
struct Line
{
in_quotes = false;
u16 next [[color("8080ff"), format("formatll")]];
if (next&0xff00 == 0)
break;
le u16 line_number [[color("00FF00")]];
LineSegment contents[while(NotZero())];
u8 eol [[color("00ffff")]];
};
Line Lines[while(!std::mem::eof())] @ 2;

View File

@@ -19,7 +19,7 @@ namespace old_binary {
};
fn format_time(u32 value) {
return std::time::format(std::time::to_utc(swap_32bit(value)));
return std::time::format(std::time::to_utc(old_binary::swap_32bit(value)));
};
using SwappedU32 = u32 [[transform("old_binary::swap_32bit"), format("old_binary::swap_32bit")]];

165
patterns/dmc3_hd_mod.hexpat Normal file
View File

@@ -0,0 +1,165 @@
#pragma description Devil May Cry 3 HD .mod 3D model file
#pragma MIME 3d-model/capcom.dmc3-hd-mod
// author = haru233, many thanks to AxCut
// ImHex Hex Pattern File for Capcom's Devil May Cry 3 HD .mod files
import std.core;
struct ModelHeader {
char ID[4];
float Version;
padding[8];
u8 objectCount;
u8 boneCount;
u8 numberTextures;
u8;
u32;
u64;
u64 skeletonOffset;
padding[24];
};
struct ObjectInfo {
u8 meshCount;
u8;
u16 numberVertices;
padding[4];
u64 meshOffset;
u32 flags;
padding[28];
float X, Y, Z;
float radius;
};
struct Positions {
float positions[3];
};
struct Normals {
float normal[3];
};
struct UVs {
s16 uv[2];
};
struct BoneIndices {
u8 boneindex[4];
};
struct Weights {
u16 weight[1];
};
struct MeshSCM {
u16 numberVertices;
u16 textureIndex;
padding[12];
u64 VerticesPositionsOffset;
u64 NormalsPositionsOffset;
u64 UVsPositionsOffset;
padding[16];
u64 unknownOffset;
u64;
padding[8];
Positions positions[numberVertices] @VerticesPositionsOffset;
Normals normals[numberVertices] @NormalsPositionsOffset;
UVs uvs[numberVertices] @UVsPositionsOffset;
};
struct Mesh {
u16 numberVertices;
u16 textureIndex;
padding[12];
u64 VerticesPositionsOffset;
u64 NormalsPositionsOffset;
u64 UVsPositionsOffset;
u64 BoneIndicesOffset;
u64 WeightsOffset;
padding[8];
u64;
padding[8];
Positions positions[numberVertices] @VerticesPositionsOffset;
Normals normals[numberVertices] @NormalsPositionsOffset;
UVs uvs[numberVertices] @UVsPositionsOffset;
BoneIndices b_index[numberVertices] @BoneIndicesOffset;
Weights weights[numberVertices] @WeightsOffset;
};
struct Hierarchy {
u8 hierarchy;
};
struct HierarchyOrder {
u8 hierarchyorder;
};
struct Unknown {
u8;
};
struct Transform {
float x;
float y;
float z;
float length; // sqrt(x*x + y*y + z*z)
padding[16];
};
struct Skeleton{
u32 hierarchyOffset;
u32 hierarchyOrderOffset;
u32 unknownOffset;
u32 transformsOffset;
};
ModelHeader modelheader @ 0x00;
ObjectInfo objects_info[modelheader.objectCount] @ 0x40;
u32 objectOffset;
struct Object {
u64 i = std::core::array_index();
if (modelheader.ID == "SCM ") {
objectOffset = objects_info[0].meshOffset;
MeshSCM meshscm[objects_info[i].meshCount] @ objects_info[i].meshOffset;
} else {
objectOffset = objects_info[0].meshOffset;
Mesh mesh[objects_info[i].meshCount] @ objects_info[i].meshOffset;
}
};
Object objects[modelheader.objectCount] @objectOffset;
Skeleton skeleton @modelheader.skeletonOffset;
Hierarchy hierarchy[modelheader.boneCount] @(modelheader.skeletonOffset + skeleton.hierarchyOffset);
HierarchyOrder hierarchyorder[modelheader.boneCount] @(modelheader.skeletonOffset + skeleton.hierarchyOrderOffset);
Unknown unknown[modelheader.boneCount] @(modelheader.skeletonOffset + skeleton.unknownOffset);
Transform transform[modelheader.boneCount] @(modelheader.skeletonOffset + skeleton.transformsOffset);

View File

@@ -0,0 +1,921 @@
/*
References:
.NET BinaryFormatter Specification "MS-NRBF":
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-nrbf/75b9fe09-be15-475f-85b8-ae7b7558cfe5
.NET runtime:
https://github.com/dotnet/runtime/blob/v8.0.17/src/libraries/System.Runtime.Serialization.Formatters/src/System/Runtime/Serialization/Formatters/Binary/BinaryParser.cs
.NET Library for Parsing MS-NRBF streams:
https://github.com/bbowyersmyth/BinaryFormatDataStructure
*/
#pragma author ODeux
#pragma description .NET BinaryFormatter (System.Runtime.Serialization.Formatters.Binary, obsolete)
#pragma endian little
import std.core;
import std.sys;
import std.mem;
import std.ptr;
fn offsetOf(ref auto value, ref auto value_field){
return addressof(value_field) - addressof(value);
};
struct NullableArrayPtr<T, pointerSize, auto size>{
pointerSize pointerValue [[no_unique_address, hidden]];
if(pointerValue != 0)
T *data[size]: pointerSize;
else
padding[sizeof(pointerSize)];
};
using _Ptr<T, PointerSize> = std::ptr::NullablePtr<T, PointerSize>;
using _ArrayPtr<T, PointerSize, auto size> = NullableArrayPtr<T, PointerSize, size>;
fn append_value_to_section(ref auto value, std::mem::Section section){
u128 old_section_size = std::mem::get_section_size(section);
std::mem::copy_value_to_section(value, section, old_section_size);
return old_section_size;
};
fn todo(auto message){
std::error(std::format("@0x{:08X} TODO: " + message, $));
};
using _Trackers;
std::mem::Section _TrackersSection = std::mem::create_section("_TrackersSection");
_Trackers _trackers @ 0x0 in _TrackersSection;
bool NeedUpdateTrackers = false;
bool IsUpdatingTrackers = false;
enum _ObjEnum: u64{
Empty = 0,
SerializedStreamHeader = 1,
ClassWithId = 2, Object = _ObjEnum::ClassWithId,
SystemClassWithMembers = 3, ObjectWithMap = _ObjEnum::SystemClassWithMembers,
ClassWithMembers = 4, ObjectWithMapAssemId = _ObjEnum::ClassWithMembers,
SystemClassWithMembersAndTypes = 5, ObjectWithMapTyped = _ObjEnum::SystemClassWithMembersAndTypes,
ClassWithMembersAndTypes = 6, ObjectWithMapTypedAssemId = _ObjEnum::ClassWithMembersAndTypes,
BinaryObjectString = 7, ObjectString = _ObjEnum::BinaryObjectString,
BinaryArray = 8, Array = _ObjEnum::BinaryArray,
MemberPrimitiveTyped = 9,
MemberReference = 10,
BinaryLibrary = 11, Assembly = _ObjEnum::BinaryLibrary,
ObjectNullMultiple256 = 12,
ObjectNullMultiple = 13,
ArraySinglePrimitive = 14,
ArraySingleObject = 15,
ArraySingleString = 16,
CrossAppDomainMap = 17,
CrossAppDomainString = 18,
CrossAppDomainAssembly = 19,
MethodCall = 20,
MethodReturn = 21
};
fn zeroedCurrObjTrackers(){
_trackers.currentObj.TypeName.pointerValue = 0;
_trackers.currentObj.AssemblyName.pointerValue = 0;
_trackers.currentObj.objEnum = _ObjEnum::Empty;
_trackers.currentObj.RawPtr = 0;
};
fn copyCurrObjAtIdTrackers(auto id){
NeedUpdateTrackers = true;
_trackers.objs[id].TypeName.pointerValue = _trackers.currentObj.TypeName.pointerValue;
_trackers.objs[id].AssemblyName.pointerValue = _trackers.currentObj.AssemblyName.pointerValue;
/* ! Enum does not get copied if we don't use a cast here for some reason ! */
_trackers.objs[id].objEnum = u64(_trackers.currentObj.objEnum);
_trackers.objs[id].RawPtr = _trackers.currentObj.RawPtr;
};
using BitfieldOrder = std::core::BitfieldOrder;
using TimeSpan = s64;
enum PrimitiveTypeEnum: u8{
Invalid = 0,
Boolean = 1,
Byte = 2,
Char = 3,
Currency = 4, /* Not Used in this protocol */
Decimal = 5,
Double = 6,
Int16 = 7,
Int32 = 8,
Int64 = 9,
SByte = 10,
Single = 11,
TimeSpan = 12,
DateTime = 13,
UInt16 = 14,
UInt32 = 15,
UInt64 = 16,
Null = 17,
String = 18
};
struct PrimitiveTypeEnumT<auto _primitiveTypeEnumT>{
PrimitiveTypeEnum primitiveTypeEnumT = _primitiveTypeEnumT;
PrimitiveTypeEnum primitiveTypeEnum;
if(_primitiveTypeEnumT > 0)
std::assert(primitiveTypeEnum == primitiveTypeEnumT, std::format("Expected {} but got {}", primitiveTypeEnumT, primitiveTypeEnum));
};
enum BinaryTypeEnum: u8{
Primitive = 0,
String = 1,
Object = 2,
SystemClass = 3, ObjectUrt = BinaryTypeEnum::SystemClass,
Class = 4, ObjectUser = BinaryTypeEnum::Class,
ObjectArray = 5,
StringArray = 6,
PrimitiveArray = 7
};
enum BinaryArrayTypeEnum: u8{
Single = 0,
Jagged = 1,
Rectangular = 2,
SingleOffset = 3,
JaggedOffset = 4,
RectangularOffset = 5
};
enum RecordTypeEnum: u8{
SerializedStreamHeader = 0,
ClassWithId = 1, Object = RecordTypeEnum::ClassWithId,
SystemClassWithMembers = 2, ObjectWithMap = RecordTypeEnum::SystemClassWithMembers,
ClassWithMembers = 3, ObjectWithMapAssemId = RecordTypeEnum::ClassWithMembers,
SystemClassWithMembersAndTypes = 4, ObjectWithMapTyped = RecordTypeEnum::SystemClassWithMembersAndTypes,
ClassWithMembersAndTypes = 5, ObjectWithMapTypedAssemId = RecordTypeEnum::ClassWithMembersAndTypes,
BinaryObjectString = 6, ObjectString = RecordTypeEnum::BinaryObjectString,
BinaryArray = 7, Array = RecordTypeEnum::BinaryArray,
MemberPrimitiveTyped = 8,
MemberReference = 9,
ObjectNull = 10,
MessageEnd = 11,
BinaryLibrary = 12, Assembly = RecordTypeEnum::BinaryLibrary,
ObjectNullMultiple256 = 13,
ObjectNullMultiple = 14,
ArraySinglePrimitive = 15,
ArraySingleObject = 16,
ArraySingleString = 17,
CrossAppDomainMap = 18,
CrossAppDomainString = 19,
CrossAppDomainAssembly = 20,
MethodCall = 21,
MethodReturn = 22
};
using BinaryHeaderEnum = RecordTypeEnum;
struct RecordTypeEnumT<auto _recordTypeEnumT>{
RecordTypeEnum recordTypeEnumT = _recordTypeEnumT;
RecordTypeEnum recordTypeEnum;
if(_recordTypeEnumT > 0)
std::assert(recordTypeEnum == recordTypeEnumT, std::format("Expected {} but got {}", recordTypeEnumT, recordTypeEnum));
};
bitfield MessageFlags{
bool NoArgs: 1; /* Arg Category */
bool ArgsInline: 1; /* Arg Category */
bool ArgsIsArray: 1; /* Arg Category */
bool ArgsInArray: 1; /* Arg Category */
bool NoContext: 1; /* Context Category */
bool ContextInline: 1; /* Context Category */
bool ContextInArray: 1; /* Context Category */
bool MethodSignatureInArray: 1; /* Signature Category */
bool PropertiesInArray: 1; /* Property Category */
bool NoReturnValue: 1; /* Return Category */
bool ReturnValueVoid: 1; /* Return Category */
bool ReturnValueInline: 1; /* Return Category */
bool ReturnValueInArray: 1; /* Return Category */
bool ExceptionInArray: 1; /* Exception Category */
bool GenericMethod: 1; /* Generic Category */
unsigned unused: 17;
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 32)]];
fn validate_MessageFlags(MessageFlags flags){
u8 arg_cnt = flags.NoArgs + flags.ArgsInline + flags.ArgsIsArray + flags.ArgsInArray;
u8 ctx_cnt = flags.NoContext + flags.ContextInline + flags.ContextInArray;
u8 sig_cnt = flags.MethodSignatureInArray;
u8 ret_cnt = flags.NoReturnValue + flags.ReturnValueVoid + flags.ReturnValueInline + flags.ReturnValueInArray;
u8 excep_cnt = flags.ExceptionInArray;
u8 prop_cnt = flags.PropertiesInArray;
u8 gen_cnt = flags.GenericMethod;
if(arg_cnt > 1 || ctx_cnt > 1 || sig_cnt > 1 || ret_cnt > 1 || excep_cnt > 1 || prop_cnt > 1 || gen_cnt > 1)
return -1;
if(arg_cnt != 0 && excep_cnt != 0) return -1;
if(ret_cnt != 0 && excep_cnt != 0) return -1;
if(ret_cnt != 0 && sig_cnt != 0) return -1;
if(excep_cnt != 0 && sig_cnt != 0) return -1;
return 1;
};
enum DateTimeKind: u8{
NOT_SPECIFIED = 0,
UTC = 1,
Local = 2
};
bitfield DateTime{
s64 Ticks: 62;
DateTimeKind kind: 2;
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 64)]];
struct vLength{
/*
Can't use that, it breaks when struct get re-parsed in _TrackersSection
u8 data[while(std::mem::read_unsigned($, 1) & 0x80)];
u8 last;
*/
u64 bytes [[no_unique_address, hidden]];
u8 cnt = 0;
if(bytes & 0x80){
if(bytes & 0x8000){
if(bytes & 0x800000){
if(bytes & 0x80000000){
if(bytes & 0x8000000000){
/* exceeding vLength 5 bytes, caller should crash */
cnt = 5;
}else cnt = 4;
}else cnt = 3;
}else cnt = 2;
}else cnt = 1;
}else cnt = 0;
u8 data[cnt];
u8 last;
} [[sealed, transform("LPS_Length_decode"), format("LPS_Length_decode")]];
fn LPS_Length_decode(auto Length){
u64 length = 0;
u8 i = 0;
for(i = 0, i < sizeof(Length.data), i += 1)
length |= u64(Length.data[i] & 0x7F) << i * 7;
length |= u64(Length.last) << i * 7;
return length;
};
struct LengthPrefixedString{
vLength Length;
std::assert(sizeof(Length) <= 5, "LengthPrefixedString.Length must be at most 5 bytes long");
char String[Length];
};
using Decimal = LengthPrefixedString;
struct ClassTypeInfo{
LengthPrefixedString TypeName;
s32 LibraryId;
};
struct ValueWithCode{
PrimitiveTypeEnum PrimitiveType;
match(PrimitiveType){
(PrimitiveTypeEnum::Boolean): bool Value;
(PrimitiveTypeEnum::Byte): u8 Value;
(PrimitiveTypeEnum::Char): char Value;
(PrimitiveTypeEnum::Currency): std::error("Primitive currency not used in this protocol");
(PrimitiveTypeEnum::Decimal): Decimal Value;
(PrimitiveTypeEnum::Double): double Value;
(PrimitiveTypeEnum::Int16): s16 Value;
(PrimitiveTypeEnum::Int32): s32 Value;
(PrimitiveTypeEnum::Int64): s64 Value;
(PrimitiveTypeEnum::SByte): s8 Value;
(PrimitiveTypeEnum::Single): float Value;
(PrimitiveTypeEnum::TimeSpan): TimeSpan Value;
(PrimitiveTypeEnum::DateTime): DateTime Value;
(PrimitiveTypeEnum::UInt16): u16 Value;
(PrimitiveTypeEnum::UInt32): u32 Value;
(PrimitiveTypeEnum::UInt64): u64 Value;
(PrimitiveTypeEnum::Null): {}
(PrimitiveTypeEnum::String): LengthPrefixedString Value;
(_): std::error(std::format("Unexpected {}", PrimitiveType));
}
};
struct StringValueWithCode: PrimitiveTypeEnumT<PrimitiveTypeEnum::String>{
LengthPrefixedString StringValue;
};
struct ArrayOfValueWithCode{
s32 Length;
ValueWithCode ListOfValueWithCode[Length];
};
struct ArrayInfo{
s32 ObjectId;
s32 Length;
};
struct ClassInfo{
s32 ObjectId;
LengthPrefixedString Name;
s32 MemberCount;
LengthPrefixedString MemberNames[MemberCount];
};
struct AdditionalInfo<auto _binaryTypeEnum>{
BinaryTypeEnum binaryTypeEnum = _binaryTypeEnum;
match(binaryTypeEnum){
(BinaryTypeEnum::SystemClass): /* ObjectUrt */
LengthPrefixedString String;
(BinaryTypeEnum::Class): /* ObjectUser */
ClassTypeInfo classTypeInfo;
(BinaryTypeEnum::Primitive | BinaryTypeEnum::PrimitiveArray): {
PrimitiveTypeEnum primitiveType;
std::assert(primitiveType != PrimitiveTypeEnum::Null &&
primitiveType != PrimitiveTypeEnum::String, "Must not be Null or String");
}
(BinaryTypeEnum::String | BinaryTypeEnum::Object |
BinaryTypeEnum::ObjectArray | BinaryTypeEnum::StringArray):
{/* not using continue here, need to keep array index matching */}
(_): std::error(std::format("Unrecognized {}", binaryTypeEnum));
}
};
struct MemberTypeInfo<auto MemberCount>{
BinaryTypeEnum binaryTypeEnums[MemberCount];
AdditionalInfo<binaryTypeEnums[std::core::array_index()]> additionalInfo[MemberCount];
};
struct UntypedMember<auto _className, auto classInfo>{
str className = _className;
u64 MemberCount = classInfo.MemberCount;
if(className == "System.Guid" && MemberCount == 11){
match(std::core::array_index()){
(0): s32 _a;
(1): s16 _b;
(2): s16 _c;
(3): u8 _d;
(4): u8 _e;
(5): u8 _f;
(6): u8 _g;
(7): u8 _h;
(8): u8 _i;
(9): u8 _j;
(10): u8 _k;
(_): std::error("unreachable");
}
}else if(MemberCount == 1 && classInfo.MemberNames[0].String == "value__"){
str Name = classInfo.MemberNames[0].String;
s32 member [name(Name)];
}else
std::error(std::format("Unsupported untyped member: {}", className));
};
using Record;
struct Members<auto _Name, auto memberTypeInfo>{
u64 i = std::core::array_index();
str Name = _Name;
BinaryTypeEnum binaryTypeEnum = memberTypeInfo.binaryTypeEnums[i];
if(binaryTypeEnum == BinaryTypeEnum::Primitive){
PrimitiveTypeEnum primitiveTypeEnum = memberTypeInfo.additionalInfo[i].primitiveType;
match(primitiveTypeEnum){
(PrimitiveTypeEnum::Boolean): bool Value [[name(Name)]];
(PrimitiveTypeEnum::Byte): u8 Value [[name(Name)]];
(PrimitiveTypeEnum::Char): char Value [[name(Name)]];
(PrimitiveTypeEnum::Currency): std::error("Primitive currency not used in this protocol");
(PrimitiveTypeEnum::Decimal): Decimal Value [[name(Name)]];
(PrimitiveTypeEnum::Double): double Value [[name(Name)]];
(PrimitiveTypeEnum::Int16): s16 Value [[name(Name)]];
(PrimitiveTypeEnum::Int32): s32 Value [[name(Name)]];
(PrimitiveTypeEnum::Int64): s64 Value [[name(Name)]];
(PrimitiveTypeEnum::SByte): s8 Value [[name(Name)]];
(PrimitiveTypeEnum::Single): float Value [[name(Name)]];
(PrimitiveTypeEnum::TimeSpan): TimeSpan Value [[name(Name)]];
(PrimitiveTypeEnum::DateTime): DateTime Value [[name(Name)]];
(PrimitiveTypeEnum::UInt16): u16 Value [[name(Name)]];
(PrimitiveTypeEnum::UInt32): u32 Value [[name(Name)]];
(PrimitiveTypeEnum::UInt64): u64 Value [[name(Name)]];
(PrimitiveTypeEnum::Null): {}
(PrimitiveTypeEnum::String): LengthPrefixedString Value [[name(Name)]];
(_): std::error(std::format("Unexpected {}", primitiveTypeEnum));
}
}else{
Record record [[name(Name)]];
}
};
struct ClassWithMembersAndTypes: RecordTypeEnumT<RecordTypeEnum::ClassWithMembersAndTypes>{
ClassInfo classInfo;
MemberTypeInfo<classInfo.MemberCount> memberTypeInfo;
s32 LibraryId;
Members<classInfo.MemberNames[std::core::array_index()].String, memberTypeInfo> members[classInfo.MemberCount];
};
struct ClassWithMembers: RecordTypeEnumT<RecordTypeEnum::ClassWithMembers>{
ClassInfo classInfo;
s32 LibraryId;
UntypedMember<classInfo.Name.String, classInfo> members[classInfo.MemberCount];
};
struct SystemClassWithMembersAndTypes: RecordTypeEnumT<RecordTypeEnum::SystemClassWithMembersAndTypes>{
ClassInfo classInfo;
MemberTypeInfo<classInfo.MemberCount> memberTypeInfo;
Members<classInfo.MemberNames[std::core::array_index()].String, memberTypeInfo> members[classInfo.MemberCount];
};
struct SystemClassWithMembers: RecordTypeEnumT<RecordTypeEnum::SystemClassWithMembers>{
ClassInfo classInfo;
UntypedMember<classInfo.Name.String, classInfo> members[classInfo.MemberCount];
};
struct ClassWithId: RecordTypeEnumT<RecordTypeEnum::ClassWithId>{
s32 ObjectId;
s32 MetadataId;
if(!IsUpdatingTrackers && NeedUpdateTrackers){
IsUpdatingTrackers = true;
_Trackers _trackers @ 0x0 in _TrackersSection;
IsUpdatingTrackers = false;
NeedUpdateTrackers = false;
}
match(_trackers.objs[MetadataId].objEnum){
(_ObjEnum::ClassWithMembersAndTypes): {
u32 MemberCount = _trackers.objs[MetadataId].classWithMembersAndTypes.data.classInfo.MemberCount;
Members<_trackers.objs[MetadataId].classWithMembersAndTypes.data.classInfo.MemberNames[std::core::array_index()].String, _trackers.objs[MetadataId].classWithMembersAndTypes.data.memberTypeInfo> members[MemberCount];
}
(_ObjEnum::SystemClassWithMembersAndTypes): {
u32 MemberCount = _trackers.objs[MetadataId].systemClassWithMemberAndTypes.data.classInfo.MemberCount;
Members<_trackers.objs[MetadataId].systemClassWithMemberAndTypes.data.classInfo.MemberNames[std::core::array_index()].String, _trackers.objs[MetadataId].systemClassWithMemberAndTypes.data.memberTypeInfo> members[MemberCount];
}
(_ObjEnum::SystemClassWithMembers): {
str className = _trackers.objs[MetadataId].TypeName.data.String;
u32 MemberCount = _trackers.objs[MetadataId].systemClassWithMembers.data.classInfo.MemberCount;
UntypedMember<className, _trackers.objs[MetadataId].systemClassWithMembers.data.classInfo> members[MemberCount];
}
(_ObjEnum::ClassWithMembers): {
str className = _trackers.objs[MetadataId].TypeName.data.String;
u32 MemberCount = _trackers.objs[MetadataId].classWithMembers.data.classInfo.MemberCount;
UntypedMember<className, _trackers.objs[MetadataId].classWithMembers.data.classInfo> members[MemberCount];
}
(_): std::error(std::format("Unexpected {}", _trackers.objs[MetadataId].objEnum));
}
};
struct BinaryArray: RecordTypeEnumT<RecordTypeEnum::BinaryArray>{
s32 ObjectId;
BinaryArrayTypeEnum binaryArrayTypeEnum;
s32 Rank;
s32 Length[Rank];
if(binaryArrayTypeEnum == BinaryArrayTypeEnum::SingleOffset ||
binaryArrayTypeEnum == BinaryArrayTypeEnum::JaggedOffset ||
binaryArrayTypeEnum == BinaryArrayTypeEnum::RectangularOffset)
s32 LowerBounds[Rank];
BinaryTypeEnum TypeEnum;
AdditionalInfo<TypeEnum> additionalInfo;
};
struct ArraySinglePrimitive: RecordTypeEnumT<RecordTypeEnum::ArraySinglePrimitive>{
ArrayInfo arrayInfo;
PrimitiveTypeEnum primitiveTypeEnum;
std::assert(primitiveTypeEnum != PrimitiveTypeEnum::Null &&
primitiveTypeEnum != PrimitiveTypeEnum::String, "Can't be one of those");
match(primitiveTypeEnum){
(PrimitiveTypeEnum::Boolean): bool Values[arrayInfo.Length];
(PrimitiveTypeEnum::Byte): u8 Value[arrayInfo.Length];
(PrimitiveTypeEnum::Char): char Value[arrayInfo.Length];
(PrimitiveTypeEnum::Currency): std::error("Primitive currency not used in this protocol");
(PrimitiveTypeEnum::Decimal): Decimal Value[arrayInfo.Length];
(PrimitiveTypeEnum::Double): double Value[arrayInfo.Length];
(PrimitiveTypeEnum::Int16): s16 Value[arrayInfo.Length];
(PrimitiveTypeEnum::Int32): s32 Value[arrayInfo.Length];
(PrimitiveTypeEnum::Int64): s64 Value[arrayInfo.Length];
(PrimitiveTypeEnum::SByte): s8 Value[arrayInfo.Length];
(PrimitiveTypeEnum::Single): float Value[arrayInfo.Length];
(PrimitiveTypeEnum::TimeSpan): TimeSpan Value[arrayInfo.Length];
(PrimitiveTypeEnum::DateTime): DateTime Value[arrayInfo.Length];
(PrimitiveTypeEnum::UInt16): u16 Value[arrayInfo.Length];
(PrimitiveTypeEnum::UInt32): u32 Value[arrayInfo.Length];
(PrimitiveTypeEnum::UInt64): u64 Value[arrayInfo.Length];
(PrimitiveTypeEnum::Null): {}
(PrimitiveTypeEnum::String): LengthPrefixedString Value[arrayInfo.Length];
(_): std::error(std::format("Unexpected {}", primitiveTypeEnum));
}
};
u64 ArraySinglElementSkipCount = 0;
struct ArraySingleElement<auto _recordTypeEnum>{
RecordTypeEnum recordTypeEnum = _recordTypeEnum;
if(ArraySinglElementSkipCount > 0)
ArraySinglElementSkipCount -= 1;
else{
if(recordTypeEnum == RecordTypeEnum::ArraySingleString){
Record record [[inline]];
match(record.recordTypeEnum){
(RecordTypeEnum::ObjectNullMultiple):
ArraySinglElementSkipCount = record.objectNullMultiple.NullCount;
(RecordTypeEnum::ObjectNullMultiple256):
ArraySinglElementSkipCount = record.objectNullMultiple256.NullCount;
(_): {}
}
}else{
Record record [[inline]];
if(record.recordTypeEnum == RecordTypeEnum::BinaryLibrary){
Record record2 [[inline]];
match(record2.recordTypeEnum){
(RecordTypeEnum::ObjectNullMultiple):
ArraySinglElementSkipCount = record2.objectNullMultiple.NullCount;
(RecordTypeEnum::ObjectNullMultiple256):
ArraySinglElementSkipCount = record2.objectNullMultiple256.NullCount;
(_): {}
}
}else{
match(record.recordTypeEnum){
(RecordTypeEnum::ObjectNullMultiple):
ArraySinglElementSkipCount = record.objectNullMultiple.NullCount;
(RecordTypeEnum::ObjectNullMultiple256):
ArraySinglElementSkipCount = record.objectNullMultiple256.NullCount;
(_): {}
}
}
}
}
};
struct ArraySingleObject: RecordTypeEnumT<RecordTypeEnum::ArraySingleObject>{
ArrayInfo arrayInfo;
ArraySinglElementSkipCount = 0;
ArraySingleElement<RecordTypeEnum::ArraySingleObject> records[arrayInfo.Length];
};
struct ArraySingleString: RecordTypeEnumT<RecordTypeEnum::ArraySingleString>{
ArrayInfo arrayInfo;
ArraySinglElementSkipCount = 0;
ArraySingleElement<RecordTypeEnum::ArraySingleString> records[arrayInfo.Length];
};
struct MethodReturnCallArray{
if(parent.MessageEnum.ReturnValueInArray)
ArraySingleObject ReturnValue;
if(parent.MessageEnum.ArgsInArray)
ArraySingleObject OutputArguments;
if(parent.MessageEnum.ExceptionInArray)
ArraySingleObject Exception;
if(parent.MessageEnum.ContextInArray)
ArraySingleObject CallContext;
if(parent.MessageEnum.PropertiesInArray)
ArraySingleObject MessageProperties;
};
struct MethodCallArray{
if(parent.MessageEnum.ArgsInArray)
ArraySingleObject InputArguments;
if(parent.MessageEnum.GenericMethod)
ArraySingleObject GenericTypeArguments;
if(parent.MessageEnum.MethodSignatureInArray)
ArraySingleObject MethodSignature;
if(parent.MessageEnum.ContextInArray)
ArraySingleObject CallContext;
if(parent.MessageEnum.PropertiesInArray)
ArraySingleObject MessageProperties;
};
struct BinaryMethodReturn: RecordTypeEnumT<RecordTypeEnum::MethodReturn>{
MessageFlags MessageEnum;
std::assert(validate_MessageFlags(MessageEnum) >= 0, "Validation Failed");
std::assert(!MessageEnum.MethodSignatureInArray && !MessageEnum.GenericMethod, "Can't be one of those");
if(MessageEnum.ReturnValueInline)
ValueWithCode ReturnValue;
if(MessageEnum.ContextInline)
StringValueWithCode CallContext;
if(MessageEnum.ArgsInline)
ArrayOfValueWithCode Args;
MethodReturnCallArray ReturnCallArray;
};
struct BinaryMethodCall: RecordTypeEnumT<RecordTypeEnum::MethodCall>{
MessageFlags MessageEnum;
std::assert(validate_MessageFlags(MessageEnum) >= 0, "Validation Failed");
std::assert(!MessageEnum.NoReturnValue && !MessageEnum.ReturnValueVoid &&
!MessageEnum.ReturnValueInline && !MessageEnum.ReturnValueInArray &&
!MessageEnum.ExceptionInArray, "Can't be one of those");
StringValueWithCode MethodName;
StringValueWithCode TypeName;
if(MessageEnum.ContextInline)
StringValueWithCode CallContext;
if(MessageEnum.ArgsInline)
ArrayOfValueWithCode Args;
MethodCallArray CallArray;
};
struct MemberPrimitiveTyped: RecordTypeEnumT<RecordTypeEnum::MemberPrimitiveTyped>{
PrimitiveTypeEnum primitiveTypeEnum;
std::assert(primitiveTypeEnum != PrimitiveTypeEnum::Null &&
primitiveTypeEnum != PrimitiveTypeEnum::String, "Can't be one of those");
match(primitiveTypeEnum){
(PrimitiveTypeEnum::Boolean): bool Value;
(PrimitiveTypeEnum::Byte): u8 Value;
(PrimitiveTypeEnum::Char): char Value;
(PrimitiveTypeEnum::Currency): std::error("Primitive currency not used in this protocol");
(PrimitiveTypeEnum::Decimal): Decimal Value;
(PrimitiveTypeEnum::Double): double Value;
(PrimitiveTypeEnum::Int16): s16 Value;
(PrimitiveTypeEnum::Int32): s32 Value;
(PrimitiveTypeEnum::Int64): s64 Value;
(PrimitiveTypeEnum::SByte): s8 Value;
(PrimitiveTypeEnum::Single): float Value;
(PrimitiveTypeEnum::TimeSpan): TimeSpan Value;
(PrimitiveTypeEnum::DateTime): DateTime Value;
(PrimitiveTypeEnum::UInt16): u16 Value;
(PrimitiveTypeEnum::UInt32): u32 Value;
(PrimitiveTypeEnum::UInt64): u64 Value;
(_): std::error(std::format("Unexpected {}", primitiveTypeEnum));
}
};
struct MemberPrimitiveUnTyped<auto PrimitiveType>{
match(PrimitiveType){
(PrimitiveTypeEnum::Boolean): bool Value;
(PrimitiveTypeEnum::Byte): u8 Value;
(PrimitiveTypeEnum::Char): char Value;
(PrimitiveTypeEnum::Currency): std::error("Primitive currency not used in this protocol");
(PrimitiveTypeEnum::Decimal): Decimal Value;
(PrimitiveTypeEnum::Double): double Value;
(PrimitiveTypeEnum::Int16): s16 Value;
(PrimitiveTypeEnum::Int32): s32 Value;
(PrimitiveTypeEnum::Int64): s64 Value;
(PrimitiveTypeEnum::SByte): s8 Value;
(PrimitiveTypeEnum::Single): float Value;
(PrimitiveTypeEnum::TimeSpan): TimeSpan Value;
(PrimitiveTypeEnum::DateTime): DateTime Value;
(PrimitiveTypeEnum::UInt16): u16 Value;
(PrimitiveTypeEnum::UInt32): u32 Value;
(PrimitiveTypeEnum::UInt64): u64 Value;
(PrimitiveTypeEnum::Null): {}
(PrimitiveTypeEnum::String):std::error("Can't be String");
(_): std::error(std::format("Unexpected {}", PrimitiveType));
}
};
struct MemberReference: RecordTypeEnumT<RecordTypeEnum::MemberReference>{
s32 IdRef;
};
struct ObjectNull: RecordTypeEnumT<RecordTypeEnum::ObjectNull>{
};
struct ObjectNullMultiple: RecordTypeEnumT<RecordTypeEnum::ObjectNullMultiple>{
s32 NullCount;
};
struct ObjectNullMultiple256: RecordTypeEnumT<RecordTypeEnum::ObjectNullMultiple256>{
u8 NullCount;
};
struct BinaryObjectString: RecordTypeEnumT<RecordTypeEnum::BinaryObjectString>{
s32 ObjectId;
LengthPrefixedString Value;
};
struct SerializationHeaderRecord: RecordTypeEnumT<RecordTypeEnum::SerializedStreamHeader>{
s32 RootId;
s32 HeaderId;
s32 MajorVersion;
std::assert_warn(MajorVersion == 1, "MajorVersion should be 1");
s32 MinorVersion;
std::assert_warn(MinorVersion == 0, "MinorVersion should be 0");
};
struct BinaryLibrary: RecordTypeEnumT<RecordTypeEnum::BinaryLibrary>{
s32 LibraryId;
LengthPrefixedString LibraryName;
};
struct MessageEnd: RecordTypeEnumT<RecordTypeEnum::MessageEnd>{
};
struct CrossAppDomainMap: RecordTypeEnumT<RecordTypeEnum::CrossAppDomainMap>{
s32 crossAppDomainArrayIndex;
};
struct CrossAppDomainString: RecordTypeEnumT<RecordTypeEnum::CrossAppDomainString>{
s32 ObjectId;
s32 Value;
};
struct CrossAppDomainAssembly: RecordTypeEnumT<RecordTypeEnum::CrossAppDomainAssembly>{
s32 LibraryId;
s32 LibraryIndex;
};
struct Record{
RecordTypeEnum recordTypeEnum [[no_unique_address, hidden]];
match(recordTypeEnum){
(RecordTypeEnum::SerializedStreamHeader):
std::error("SerializationStreamHeader can only appear at the top of the document");
(RecordTypeEnum::ClassWithId): { /* RecordTypeEnum::Object */
ClassWithId classWithId;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::ClassWithId;
_trackers.currentObj.RawPtr = append_value_to_section(classWithId, _TrackersSection);
_trackers.currentObj.TypeName.pointerValue = _trackers.objs[classWithId.MetadataId].TypeName.pointerValue;
_trackers.currentObj.AssemblyName.pointerValue =_trackers.objs[classWithId.MetadataId].AssemblyName.pointerValue ;
if(classWithId.ObjectId != 0)
copyCurrObjAtIdTrackers(classWithId.ObjectId);
}
}
(RecordTypeEnum::SystemClassWithMembers): { /* RecordTypeEnum::ObjectWithMap */
SystemClassWithMembers systemClassWithMembers;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::SystemClassWithMembers;
_trackers.currentObj.RawPtr = append_value_to_section(systemClassWithMembers, _TrackersSection);
_trackers.currentObj.TypeName.pointerValue = _trackers.currentObj.RawPtr + offsetOf(systemClassWithMembers, systemClassWithMembers.classInfo.Name);
if(systemClassWithMembers.classInfo.ObjectId != 0)
copyCurrObjAtIdTrackers(systemClassWithMembers.classInfo.ObjectId);
}
}
(RecordTypeEnum::ClassWithMembers): { /* RecordTypeEnum::ObjectWithMapAssemId */
ClassWithMembers classWithMembers;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::ClassWithMembers;
_trackers.currentObj.RawPtr = append_value_to_section(classWithMembers, _TrackersSection);
_trackers.currentObj.TypeName.pointerValue = _trackers.currentObj.RawPtr + offsetOf(classWithMembers, classWithMembers.classInfo.Name);
_trackers.currentObj.AssemblyName.pointerValue = _trackers.libs[classWithMembers.LibraryId].pointerValue;
if(classWithMembers.classInfo.ObjectId != 0)
copyCurrObjAtIdTrackers(classWithMembers.classInfo.ObjectId);
}
}
(RecordTypeEnum::SystemClassWithMembersAndTypes): { /* RecordTypeEnum::ObjectWithMapTyped */
SystemClassWithMembersAndTypes systemClassWithMembersAndTypes;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::SystemClassWithMembersAndTypes;
_trackers.currentObj.RawPtr = append_value_to_section(systemClassWithMembersAndTypes, _TrackersSection);
_trackers.currentObj.TypeName.pointerValue = _trackers.currentObj.RawPtr + offsetOf(systemClassWithMembersAndTypes, systemClassWithMembersAndTypes.classInfo.Name);
if(systemClassWithMembersAndTypes.classInfo.ObjectId != 0)
copyCurrObjAtIdTrackers(systemClassWithMembersAndTypes.classInfo.ObjectId);
}
}
(RecordTypeEnum::ClassWithMembersAndTypes): { /* RecordTypeEnum::ObjectWithMapTypedAssemId */
ClassWithMembersAndTypes classWithMembersAndTypes;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::ClassWithMembersAndTypes;
_trackers.currentObj.RawPtr = append_value_to_section(classWithMembersAndTypes, _TrackersSection);
_trackers.currentObj.TypeName.pointerValue = _trackers.currentObj.RawPtr + offsetOf(classWithMembersAndTypes, classWithMembersAndTypes.classInfo.Name);
_trackers.currentObj.AssemblyName.pointerValue = _trackers.libs[classWithMembersAndTypes.LibraryId].pointerValue;
if(classWithMembersAndTypes.classInfo.ObjectId != 0)
copyCurrObjAtIdTrackers(classWithMembersAndTypes.classInfo.ObjectId);
}
}
(RecordTypeEnum::BinaryObjectString): { /* RecordTypeEnum::ObjectString */
BinaryObjectString binaryObjectString;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::BinaryObjectString;
_trackers.currentObj.RawPtr = append_value_to_section(binaryObjectString, _TrackersSection);
if(binaryObjectString.ObjectId != 0)
copyCurrObjAtIdTrackers(binaryObjectString.ObjectId);
}
}
(RecordTypeEnum::BinaryArray): { /* RecordTypeEnum::Array */
BinaryArray binaryArray;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::BinaryArray;
_trackers.currentObj.RawPtr = append_value_to_section(binaryArray, _TrackersSection);
if(binaryArray.ObjectId != 0)
copyCurrObjAtIdTrackers(binaryArray.ObjectId);
}
}
(RecordTypeEnum::MemberPrimitiveTyped): {
MemberPrimitiveTyped memberPrimitiveTyped;
}
(RecordTypeEnum::MemberReference): {
MemberReference memberReference;
}
(RecordTypeEnum::ObjectNull): {}
(RecordTypeEnum::MessageEnd): break;
(RecordTypeEnum::BinaryLibrary): { /* RecordTypeEnum::Assembly */
BinaryLibrary library;
if(!IsUpdatingTrackers)
_trackers.libs[library.LibraryId].pointerValue = append_value_to_section(library.LibraryName, _TrackersSection);
}
(RecordTypeEnum::ObjectNullMultiple256): {
ObjectNullMultiple256 objectNullMultiple256;
}
(RecordTypeEnum::ObjectNullMultiple): {
ObjectNullMultiple objectNullMultiple;
}
(RecordTypeEnum::ArraySinglePrimitive): {
ArraySinglePrimitive arraySinglePrimitive;
if(!IsUpdatingTrackers){
zeroedCurrObjTrackers();
_trackers.currentObj.objEnum = _ObjEnum::ArraySinglePrimitive;
_trackers.currentObj.RawPtr = append_value_to_section(arraySinglePrimitive, _TrackersSection);
if(arraySinglePrimitive.arrayInfo.ObjectId != 0)
copyCurrObjAtIdTrackers(arraySinglePrimitive.arrayInfo.ObjectId);
}
}
(RecordTypeEnum::ArraySingleObject): {
ArraySingleObject arraySingleObject;
}
(RecordTypeEnum::ArraySingleString): {
ArraySingleString arraySingleString;
}
(RecordTypeEnum::CrossAppDomainMap):
CrossAppDomainMap crossAppDomainMap;
(RecordTypeEnum::CrossAppDomainString):
CrossAppDomainString crossAppDomainString;
(RecordTypeEnum::CrossAppDomainAssembly):
CrossAppDomainAssembly crossAppDomainAssembly;
(RecordTypeEnum::MethodCall): {
BinaryMethodCall binaryMethodCall;
}
(RecordTypeEnum::MethodReturn): {
BinaryMethodReturn binaryMethodReturn;
}
(_): std::error(std::format("Unrecognized {}", recordTypeEnum));
}
};
struct DotNetBinnaryFormatter{
SerializationHeaderRecord Header;
Record records[while(std::mem::read_unsigned($, 1) != RecordTypeEnum::MessageEnd)];
MessageEnd End;
if(!IsUpdatingTrackers && NeedUpdateTrackers){
IsUpdatingTrackers = true;
_Trackers _trackers @ 0x0 in _TrackersSection;
IsUpdatingTrackers = false;
NeedUpdateTrackers = false;
}
};
struct _ObjTracker{
_Ptr<LengthPrefixedString, u64> TypeName;
_Ptr<LengthPrefixedString, u64> AssemblyName;
_ObjEnum objEnum;
u64 RawPtr [[no_unique_address]];
/* Only enable the one we actually use to avoid significant slow down */
match(objEnum){
(_ObjEnum::Empty):
u64 ptr;
/*
(_ObjEnum::ClassWithId): // _ObjEnum::Object
_Ptr<ClassWithId, u64> classWithId;
*/
(_ObjEnum::SystemClassWithMembers): // _ObjEnum::ObjectWithMap
_Ptr<SystemClassWithMembers, u64> systemClassWithMembers;
(_ObjEnum::ClassWithMembers): // _ObjEnum::ObjectWithMapAssemId
_Ptr<ClassWithMembers, u64> classWithMembers;
(_ObjEnum::SystemClassWithMembersAndTypes): // _ObjEnum::ObjectWithMapTyped
_Ptr<SystemClassWithMembersAndTypes, u64> systemClassWithMemberAndTypes;
(_ObjEnum::ClassWithMembersAndTypes): // _ObjEnum::ObjectWithMapTypedAssemId
_Ptr<ClassWithMembersAndTypes, u64> classWithMembersAndTypes;
/*
(_ObjEnum::BinaryObjectString): // _ObjEnum::ObjectString
_Ptr<BinaryObjectString, u64> binaryObjectString;
(_ObjEnum::BinaryArray): // _ObjEnum::Array
_Ptr<BinaryArray, u64> binaryArray;
(_ObjEnum::MemberPrimitiveTyped):
_Ptr<MemberPrimitiveTyped, u64> memberPrimitiveTyped;
(_ObjEnum::MemberReference):
_Ptr<MemberReference, u64> memberReference;
(_ObjEnum::BinaryLibrary): // _ObjEnum::Assembly
_Ptr<BinaryLibrary, u64> library;
(_ObjEnum::ObjectNullMultiple256):
_Ptr<ObjectNullMultiple256, u64> objectNullMultiple256;
(_ObjEnum::ObjectNullMultiple):
_Ptr<ObjectNullMultiple, u64> objectNullMultiple;
(_ObjEnum::ArraySinglePrimitive):
_Ptr<ArraySinglePrimitive, u64> arraySinglePrimitive;
(_ObjEnum::ArraySingleObject):
_Ptr<ArraySingleObject, u64> arraySingleObject;
(_ObjEnum::ArraySingleString):
_Ptr<ArraySingleString, u64> arraySingleString;
(_ObjEnum::CrossAppDomainMap):
_Ptr<CrossAppDomainMap, u64> crossAppDomainMap;
(_ObjEnum::CrossAppDomainString):
_Ptr<CrossAppDomainString, u64> crossAppDomainString;
(_ObjEnum::CrossAppDomainAssembly):
_Ptr<CrossAppDomainAssembly, u64> crossAppDomainAssembly;
(_ObjEnum::MethodCall):
_Ptr<BinaryMethodCall, u64> binaryMethodCall;
(_ObjEnum::MethodReturn):
_Ptr<BinaryMethodReturn, u64> binaryMethodReturn;
*/
(_): u64 ptr; //std::error(std::format("Unexpected {}", objEnum));
}
};
/* Should be an In variable with default non zero value in the future until we use a hash map */
#define _OBJECTS_TRACKER_CNT 232
#define _LIBRARIES_CNT 8
struct _Trackers{
_ObjTracker currentObj;
/* TODO: this should really be an hash map, the algorithm that generated is unspecified */
_ObjTracker objs[_OBJECTS_TRACKER_CNT];
_Ptr<LengthPrefixedString, u64> libs[_LIBRARIES_CNT];
};
std::mem::set_section_size(_TrackersSection, sizeof(_trackers));
DotNetBinnaryFormatter dotNetBinnaryFormatter @ 0x0;

View File

@@ -497,14 +497,15 @@ bitfield SHF {
};
bitfield ELF32_R_INFO {
SYM : 8;
TYPE : 8;
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 16)]];
TYPE : 8;
SYM : 8;
padding : 16;
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 32)]];
bitfield ELF64_R_INFO {
SYM : 32;
TYPE : 32;
} [[bitfield_order(BitfieldOrder::MostToLeastSignificant, 64)]];
SYM : 32;
} [[bitfield_order(BitfieldOrder::LeastToMostSignificant, 64)]];
bitfield PF {
X : 1;
@@ -643,9 +644,12 @@ struct Elf32_Shdr {
// Section has no data
} else if (sh_type == SHT::STRTAB) {
String stringTable[while($ < (sh_offset + sh_size))] @ sh_offset;
stringTableIndex = std::core::array_index();
} else if (sh_type == SHT::SYMTAB || sh_type == SHT::DYNSYM) {
Elf32_Sym symbolTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::REL) {
Elf32_Rel relTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::RELA) {
Elf32_Rela relaTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::INIT_ARRAY || sh_type == SHT::FINI_ARRAY) {
u32 pointer[while($ < (sh_offset + sh_size))] @ sh_offset;
} else {
@@ -703,9 +707,12 @@ struct Elf64_Shdr {
// Section has no data
} else if (sh_type == SHT::STRTAB) {
String stringTable[while($ < (sh_offset + sh_size))] @ sh_offset;
stringTableIndex = std::core::array_index();
} else if (sh_type == SHT::SYMTAB || sh_type == SHT::DYNSYM) {
Elf64_Sym symbolTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::REL) {
Elf64_Rel relTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::RELA) {
Elf64_Rela relaTable[sh_size / sh_entsize] @ sh_offset;
} else if (sh_type == SHT::INIT_ARRAY || sh_type == SHT::FINI_ARRAY) {
u32 pointer[while($ < (sh_offset + sh_size))] @ sh_offset;
} else {
@@ -734,17 +741,24 @@ struct ELF {
if (e_ident.EI_CLASS == EI_CLASS::ELFCLASS32) {
Elf32_Ehdr ehdr;
stringTableIndex = ehdr.e_shstrndx;
Elf32_Phdr phdr[ehdr.e_phnum] @ ehdr.e_phoff;
Elf32_Shdr shdr[ehdr.e_shnum] @ ehdr.e_shoff;
} else if (e_ident.EI_CLASS == EI_CLASS::ELFCLASS64) {
Elf64_Ehdr ehdr;
stringTableIndex = ehdr.e_shstrndx;
Elf64_Phdr phdr[ehdr.e_phnum] @ ehdr.e_phoff;
Elf64_Shdr shdr[ehdr.e_shnum] @ ehdr.e_shoff;
}
};
ELF elf @ 0x00;
EI_DATA endian @ 0x05 [[hidden]];
match (endian) {
(EI_DATA::ELFDATA2LSB): std::core::set_endian(std::mem::Endian::Little);
(EI_DATA::ELFDATA2MSB): std::core::set_endian(std::mem::Endian::Big);
(_): std::core::set_endian(std::mem::Endian::Native);
}
fn gen_shdr_disp_name(ref auto pattern, str member_name, u8 member_index) {
return std::format(

View File

@@ -1,25 +1,25 @@
#pragma author DmitriLeon2000
#pragma description Oska Software DeskMates FAS (Frames and Sequences) file
#pragma description Oska Software DeskMates FAS (Frames and Sequences)
#pragma endian little
enum Compression : u32 {
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
};
struct Colors {
u8 blue;
u8 green;
u8 red;
u8 reserved;
u8 blue;
u8 green;
u8 red;
u8 reserved;
};
struct FASHeader {
@@ -69,7 +69,7 @@ struct SeqHeader {
};
struct Bitmap {
u8 byte[fas.fasHeader.version >= 3 ? fas.fasHeader.frameSize + (fas.frameSizeHigh << 16) : fas.fasHeader.frameSize];
u8 byte[parent.fasHeader.version >= 3 ? parent.fasHeader.frameSize + (parent.frameSizeHigh << 16) : parent.fasHeader.frameSize] [[sealed]];
};
struct FramesHeader {
@@ -105,7 +105,7 @@ struct FAS {
u8 filenameChecksum; // a checksum for a filename in ASCII
AnimSequence sequences[seqHeader.count];
FramesHeader framesHeader;
Bitmap framesBitmap[framesHeader.count];
};
FAS fas @ 0x00;
Bitmap framesBitmap[fas.framesHeader.count] @ $;

View File

@@ -1,25 +1,25 @@
#pragma author DmitriLeon2000
#pragma description Oska Software DeskMates FAS (Frames and Sequences) file (Oska DeskMate versions 1.3 and 2.06)
#pragma description Oska Software DeskMates FAS (Frames and Sequences) (Oska DeskMate 1.03 and 2.06)
#pragma endian little
enum Compression : u32 {
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
BI_RGB,
BI_RLE8,
BI_RLE4,
BI_BITFIELDS,
BI_JPEG,
BI_PNG,
BI_ALPHABITFIELDS,
BI_CMYK,
BI_CMYKRLE8,
BI_CMYKRLE4,
};
struct Colors {
u8 blue;
u8 green;
u8 red;
u8 reserved;
u8 blue;
u8 green;
u8 red;
u8 reserved;
};
struct FASHeader {
@@ -59,14 +59,14 @@ struct AnimSequence {
};
struct SeqHeader {
le u32 size;
le u32 count;
le u32 strPointers[count*2];
le u32 size;
le u32 count;
le u32 strPointers[count*2];
};
struct Frame {
u8 colorBitmap[fas.fasHeader.frameSizeColor];
u8 maskBitmap[fas.fasHeader.frameSizeMask];
u8 colorBitmap[parent.fasHeader.frameSizeColor] [[sealed]];
u8 maskBitmap[parent.fasHeader.frameSizeMask] [[sealed]];
};
struct FramesHeader {
@@ -80,8 +80,7 @@ struct FAS {
SeqHeader seqHeader;
AnimSequence sequences[seqHeader.count];
FramesHeader framesHeader;
Frame frames[framesHeader.count];
};
FAS fas @ 0x00;
Frame frames[fas.framesHeader.count] @ $;

268
patterns/flc.hexpat Normal file
View File

@@ -0,0 +1,268 @@
#pragma description FLC/FLIC animation file
#pragma endian little
#pragma magic [12 AF] @ 0x4
// A flic file could contain many DELTA_FLC chunks, which contain many RLE packets
//#define DECODE_DELTA_FLC
import std.mem;
import std.core;
import std.io;
import type.color;
using Color = type::RGB8 [[hex::inline_visualize("color", r, g, b, 0xff)]];
bitfield Color64 {
r : 6;
padding : 2;
g : 6;
padding : 2;
b : 6;
padding : 2;
u8 r8 = r << 2 | r >> 4;
u8 g8 = g << 2 | g >> 4;
u8 b8 = b << 2 | b >> 4;
} [[hex::inline_visualize("color", r8, g8, b8, 0xff)]];
enum FlicType : u16 {
FLI = 0xAF11,
FLC_8bit = 0xAF12,
FLIC_other = 0xAF44,
FLIC_Huffmann = 0xAF30,
FLIC_frame_shift = 0xAF31,
};
enum ChunkType : u16 {
CEL_DATA = 3, // Registration and transparency
COLOR_256 = 4, // 256-level colour palette
DELTA_FLC = 7, // Delta image, word oriented RLE (FLI_SS2)
COLOR_64 = 11, // 64-level colour palette
DELTA_FLI = 12, // Delta image, byte oriented RLE (FLI_LC)
BLACK = 13, // Full black frame (rare)
BYTE_RUN = 15, // Full image, byte oriented RLE (FLI_BRUN)
FLI_COPY = 16, // Uncompressed image (rare)
PSTAMP = 18, // Postage stamp (icon of the first frame)
DTA_BRUN = 25, // Full image, pixel oriented RLE
DTA_COPY = 26, // Uncompressed image
DTA_LC = 27, // Delta image, pixel oriented RLE
LABEL = 31, // Frame label
BMP_MASK = 32, // Bitmap mask
MLEV_MASK = 33, // Multilevel mask
SEGMENT = 34, // Segment information
KEY_IMAGE = 35, // Key image, similar to BYTE_RUN / DTA_BRUN
KEY_PAL = 36, // Key palette, similar to COLOR_256
REGION = 37, // Region of frame differences
WAVE = 38, // Digitized audio
USERSTRING = 39, // General purpose user data
RGN_MASK = 40, // Region mask
LABELEX = 41, // Extended frame label (includes symbolic name)
SHIFT = 42, // Scanline delta shifts (compression)
PATHMAP = 43, // Path map (segment transitions)
PREFIX_TYPE = 0xF100, // Prefix chunk
SCRIPT_CHUNK = 0xF1E0, // Embedded "Small" script
FRAME_TYPE = 0xF1FA, // Frame chunk
SEGMENT_TABLE = 0xF1FB, // Segment table chunk
HUFFMAN_TABLE = 0xF1FC // Huffman compression table chunk
};
bitfield ExtFlags {
segment_table_present : 1;
regular_key_imagees : 1;
identical_palette : 1;
huffmann_compressed : 1;
bwt_huffmann_compressed : 1;
bitmap_masks_present : 1;
multilevel_masks_present : 1;
region_data_present : 1;
password_protected : 1;
digitized_audio : 1;
contains_script : 1;
region_masks_present : 1;
contains_overlays : 1;
frame_shift_compressed : 1;
padding : 2;
};
u16 pixels_per_line = 0;
u16 lines = 0;
struct Header {
u32 size [[comment("Size of FLIC including this header")]];
FlicType type [[comment("File type 0xAF11, 0xAF12, 0xAF30, 0xAF44, ...")]];
u16 frames [[comment("Number of frames in first segment")]];
u16 width [[comment("FLIC width in pixels")]];
u16 height [[comment("FLIC height in pixels")]];
u16 depth [[comment("Bits per pixel (usually 8)")]];
u16 flags [[comment("Set to zero or to three")]];
u32 speed [[comment("Delay between frames")]];
u16 reserved1 [[comment("Set to zero")]];
u32 created [[comment("Date of FLIC creation (FLC only)")]];
u32 creator [[comment("Serial number or compiler id (FLC only)")]];
u32 updated [[comment("Date of FLIC update (FLC only)")]];
u32 updater [[comment("Serial number (FLC only), see creator")]];
u16 aspect_dx [[comment("Width of square rectangle (FLC only)")]];
u16 aspect_dy [[comment("Height of square rectangle (FLC only)")]];
ExtFlags ext_flags [[comment("EGI: flags for specific EGI extensions")]];
u16 keyframes [[comment("EGI: key-image frequency")]];
u16 totalframes [[comment("EGI: total number of frames (segments)")]];
u32 req_memory [[comment("EGI: maximum chunk size (uncompressed)")]];
u16 max_regions [[comment("EGI: max. number of regions in a CHK_REGION chunk")]];
u16 transp_num [[comment("EGI: number of transparent levels")]];
u8 reserved2[24] [[comment("Set to zero")]];
u32 oframe1 [[comment("Offset to frame 1 (FLC only)")]];
u32 oframe2 [[comment("Offset to frame 2 (FLC only)")]];
u8 reserved3[40] [[comment("Set to zero")]];
pixels_per_line = width;
lines = height;
};
namespace DeltaFLCChunk {
enum OpcodeType : u8 {
Count = 0b00,
Undefined = 0b01,
LowByte = 0b10,
LineSkipCount = 0b11,
};
struct Packet {
u8 skip_count;
s8 count;
if (count < 0) {
u8 replicate_bytes[2];
} else {
u8 literal_bytes[count * 2];
}
};
u16 packets_in_line;
fn format_opcode(auto opcode) {
match (opcode.type) {
(OpcodeType::Count): return std::format("Packet(s): {0}", opcode.value);
(OpcodeType::Undefined): return "Undefined";
(OpcodeType::LowByte): return std::format("Last byte: {0:02x}", opcode.value);
(OpcodeType::LineSkipCount): {
s16 total = 0xC000 | opcode.value;
return std::format("Lines skipped: {0}", total);
}
}
};
bitfield Opcode {
value: 14;
OpcodeType type: 2;
if (type == OpcodeType::Count) {
packets_in_line = value;
break;
}
} [[format("DeltaFLCChunk::format_opcode")]];
struct Line {
packets_in_line = 0;
Opcode opcodes[while(true)];
Packet packets[packets_in_line];
};
}
namespace ByteRunChunk {
u16 pixels_in_line_counter = 0;
struct Packet {
s8 count;
if (count < 0) {
pixels_in_line_counter = -count + pixels_in_line_counter;
u8 literal_bytes[-count];
} else {
pixels_in_line_counter = count + pixels_in_line_counter;
u8 replicate_byte;
}
} [[format("format_byte_run_packet")]];
struct Line {
pixels_in_line_counter = 0;
u8 packet_count;
Packet packets[while(pixels_in_line_counter < pixels_per_line)];
};
}
struct ColorPalettePacket<C> {
u8 skip;
u8 copy;
if (copy == 0) {
C color[256];
} else {
C color[copy];
}
};
fn format_chunk(auto chunk) {
return std::format("{}", chunk.type);
};
fn format_byte_run_packet(auto packet) {
if (packet.count == -1) {
return std::format("One {}", packet.literal_bytes[0]);
} else if (packet.count < 0) {
return std::format("{} bytes", -packet.count);
} else {
return std::format("{}x color {}", packet.count, packet.replicate_byte);
}
};
struct Chunk {
u32 size;
ChunkType type;
if (type == ChunkType::FRAME_TYPE) {
u16 chunks;
u16 delay;
u16 reserved;
u16 width;
u16 height;
// Not sure if this is the intended operation here?
if (width > 0) {
pixels_per_line = width;
}
if (height > 0) {
lines = height;
}
Chunk subchunks[chunks];
} else if (type == ChunkType::COLOR_256) {
u16 num_packets;
ColorPalettePacket<Color> packets[num_packets];
} else if (type == ChunkType::COLOR_64) {
u16 num_packets;
ColorPalettePacket<Color64> packets[num_packets];
} else if (type == ChunkType::BYTE_RUN) {
ByteRunChunk::Line lines[lines];
} else if (type == ChunkType::DELTA_FLC) {
u16 line_count;
#ifdef DECODE_DELTA_FLC
DeltaFLCChunk::Line lines[line_count];
#endif
#ifndef DECODE_DELTA_FLC
u8 data[size - ($ - addressof(this))];
#endif
} else if (type == ChunkType::PSTAMP) {
u16 height;
u16 width;
u16 xlate;
u8 data[size - ($ - addressof(this))];
} else if (type == ChunkType::FLI_COPY) {
u8 pixels[lines * pixels_per_line];
}
s128 remainingBytes = size - ($ - addressof(this));
// When the chunk size is rounded up to the next even number, it might need a padding byte.
// Unrecognized chunk types will also fill out the padding
if (remainingBytes > 0) {
padding[size - ($ - addressof(this))];
} else if (remainingBytes < 0) {
std::warning("Chunk size wasn't large enough");
}
} [[format("format_chunk")]];
Header header @ $;
Chunk chunks[while(!std::mem::eof())] @ $;

224
patterns/flv.hexpat Normal file
View File

@@ -0,0 +1,224 @@
#pragma author ZN123 version 1.0
#pragma description flv Archive
#pragma MIME video/x-flv
#pragma magic [46 4C 56] @ 0x00
// zn123@sina.com
// https://veovera.org/docs/enhanced/enhanced-rtmp-v2#flv-file-format-overview
import std.io;
import type.base;
import std.mem;
import std.math;
import std.string;
#pragma endian big
#pragma pattern_limit 2000000
enum TypeB:u24{
flvSignature = 0x564C46, // Defining flv signature
};
#define FLV_HEADER_FLAG_HASVIDEO 1
#define FLV_HEADER_FLAG_HASAUDIO 4
struct FLVHeader {
char signature[3] [[color("FF0000")] ];
u8 version [[color("00FF00")]];
u8 stream_info [[color("0000FF")]]; //si->missing_streams = flags & (FLV_HEADER_FLAG_HASVIDEO | FLV_HEADER_FLAG_HASAUDIO);
if (stream_info & FLV_HEADER_FLAG_HASVIDEO)
std::print("HAS VIDEO");
if (stream_info & FLV_HEADER_FLAG_HASAUDIO)
std::print("HAS AUDIO");
u32 head_size [[color("FFFF00")]];
u32 previous_tag_size [[comment("away 0")]];
};
enum FLV_TAG_TYPE : u8 {
FLV_TAG_TYPE_AUDIO = 8,
FLV_TAG_TYPE_VIDEO = 9,
FLV_TAG_TYPE_META = 18
};
#define FLV_AUDIO_CODECID_MASK 0xf0
#define FLV_VIDEO_CODECID_MASK 0x0f
#define FLV_VIDEO_FRAMETYPE_MASK 0x70
#define FLV_VIDEO_FRAMETYPE_OFFSET 4
// type = (avio_r8(s->pb) & 0x1F);
// dts = avio_rb24(s->pb);
// dts |= (unsigned)avio_r8(s->pb) << 24;
struct BaseTag{
// if (std::mem::eof()) break;
u8 TagType;
u24 offset [[comment("offset+11 = taglen")]];
u24 Timestamp;
u8 TimestampExtended;
u24 StreamID [[comment("away 0")]];
// std::print("TagType= {}",TagType & 0x1F );
// std::print("offset= {}",offset + 11);
// std::print("dts={},{}, {}",Timestamp,TimestampExtended,Timestamp + (TimestampExtended<<3));
};
enum AMFDataType:u8 {
AMF_DATA_TYPE_NUMBER = 0x00,
AMF_DATA_TYPE_BOOL = 0x01,
AMF_DATA_TYPE_STRING = 0x02,
AMF_DATA_TYPE_OBJECT = 0x03,
AMF_DATA_TYPE_NULL = 0x05,
AMF_DATA_TYPE_UNDEFINED = 0x06,
AMF_DATA_TYPE_REFERENCE = 0x07,
AMF_DATA_TYPE_MIXEDARRAY = 0x08,
AMF_DATA_TYPE_OBJECT_END = 0x09,
AMF_DATA_TYPE_ARRAY = 0x0a,
AMF_DATA_TYPE_DATE = 0x0b,
AMF_DATA_TYPE_LONG_STRING = 0x0c,
AMF_DATA_TYPE_UNSUPPORTED = 0x0d,
};
#define TYPE_ONTEXTDATA 1
#define TYPE_ONCAPTION 2
#define TYPE_ONCAPTIONINFO 3
#define TYPE_UNKNOWN 9
fn flv_read_metabody(auto offset){
u8 flags = std::mem::read_unsigned($ , 1, std::mem::Endian::Big);
u64 startOffset = $;
u64 endOffset = $ + offset;
std::print("metabody {},{}",startOffset,endOffset);
if (flags != AMFDataType::AMF_DATA_TYPE_STRING){
return TYPE_UNKNOWN;
}
u16 len = std::mem::read_unsigned($+1 , 2, std::mem::Endian::Big);
if (len > endOffset) return 0;
str name = std::mem::read_string($+3,len);
std::print("AMF_DATA_TYPE_STRING {} {}",len,name);
if (std::string::contains(name, "onMetaData")) {
std::print("onMetaData");
} else if (std::string::contains(name, "onTextData")) {
std::print("onTextData");
} else if (std::string::contains(name, "onCaption")) {
std::print("onCaption");
} else if (std::string::contains(name, "onCaptionInfo")) {
std::print("onCaptionInfo");
} else if (std::string::contains(name, "onTextData")) {
std::print("onTextData");
} else {
std::error("Invalid DataEntryBox");
}
return flags;
};
/***
//#define FLV_AUDIO_CODECID_OFFSET 4
enum FLV_CODECID:u8{
FLV_CODECID_PCM = 0,
FLV_CODECID_ADPCM = 1 ,
FLV_CODECID_MP3 = 2 ,
FLV_CODECID_PCM_LE = 3 ,
FLV_CODECID_NELLYMOSER_16KHZ_MONO = 4 ,
FLV_CODECID_NELLYMOSER_8KHZ_MONO = 5 ,
FLV_CODECID_NELLYMOSER = 6 ,
FLV_CODECID_PCM_ALAW = 7 ,
FLV_CODECID_PCM_MULAW = 8 ,
FLV_CODECID_OPUS = 9 ,//china//1001
FLV_CODECID_AAC = 10,//1010
FLV_CODECID_SPEEX = 11,//11<< FLV_AUDIO_CODECID_OFFSET,1011
};
***/
bitfield FLVTagAudio_flags {
audio_codecid : 4;
sample_rate : 2; //3:44100 2:22k 1:11 0:5.5 [AAC away 3]
channels : 1 [[format("format_channels")]]; //0 mono ,1 stereo [AAC away 1]
bits_per_coded_sample : 1; //0 8bit,1 16bit
} [[inline]];
fn format_channels(u8 value) {
return value;
};
struct FLVTagAudio{
FLVTagAudio_flags flags;
u8 data[parent.offset-1] [[sealed]];
};
/***
enum FrameType:u8{
FLV_FRAME_KEY = 1 , ///< key frame (for AVC, a seekable frame) 001
FLV_FRAME_INTER = 2 , ///< inter frame (for AVC, a non-seekable frame) 010
FLV_FRAME_DISP_INTER = 3 , ///< disposable inter frame (H.263 only) 011
FLV_FRAME_GENERATED_KEY = 4 , ///< generated key frame (reserved for server use only) 100
FLV_FRAME_VIDEO_INFO_CMD = 5 , ///< video info/command frame 101
};
enum {
FLV_CODECID_H263 = 2,
FLV_CODECID_SCREEN = 3,
FLV_CODECID_VP6 = 4,
FLV_CODECID_VP6A = 5,
FLV_CODECID_SCREEN2 = 6,
FLV_CODECID_H264 = 7,
FLV_CODECID_REALH263= 8,
FLV_CODECID_MPEG4 = 9,
FLV_CODECID_HEVC = 12, //china
FLV_CODECID_AV1 = 13, //china
FLV_CODECID_VP8 = 14, //china
FLV_CODECID_VP9 = 15, //china
};
enum {
PacketTypeSequenceStart = 0,
PacketTypeCodedFrames = 1,
PacketTypeSequenceEnd = 2,
PacketTypeCodedFramesX = 3,
PacketTypeMetadata = 4,
PacketTypeMPEG2TSSequenceStart = 5,
};
***/
bitfield FLVTagVideo_flags {
enhanced_flv : 1;
FrameType : 3;
video_codecid : 4;
} [[inline]];
bitfield FLVTagVideoEXT_flags {
enhanced_flv : 1;
FrameType : 3;
PacketType : 4;
} [[inline]];
struct FLVTagVideo{
FLVTagVideo_flags flags;
u8 PacketType;
u8 data[parent.offset-1-1] [[sealed]];
};
struct FLVTagVideoExt{
FLVTagVideoEXT_flags flags;
char video_codecid[4];
u8 data[parent.offset-1-4] [[sealed]];
};
struct FLVTag: BaseTag{
try {
u8 type =(TagType & 0x1F);
if (type == FLV_TAG_TYPE::FLV_TAG_TYPE_META){
std::print("___FLV_TAG_TYPE::FLV_TAG_TYPE_META");
// flv_read_metabody(offset);
u8 data[offset] [[sealed]];
}else if (type == FLV_TAG_TYPE::FLV_TAG_TYPE_VIDEO){
u8 ext = std::mem::read_unsigned($ , 1, std::mem::Endian::Big);
ext = (ext & 0x80) >> 7;
if (ext == 1){
FLVTagVideoExt tagVideoExt;
}else{
FLVTagVideo tagVideo;
}
}else if (type == FLV_TAG_TYPE::FLV_TAG_TYPE_AUDIO){
//u8 data[offset] [[sealed]];
FLVTagAudio tagAudio;
}else{
std::print("___FLV_TAG_TYPE::UNKOWN");
u8 data[offset] [[sealed]];
}
u32 previous_tag_size;
} catch {
std::warning("Invalid sample payload at chunk");
break;
}
} [[name(std::format("FLVTag({})", TagType))]];
struct FLV {
FLVHeader flvheader;
FLVTag flvtag[while(!std::mem::eof())];
};
FLV flv @ 0x00 [[inline]];

311
patterns/fs/exfat.hexpat Normal file
View File

@@ -0,0 +1,311 @@
#pragma magic [45 58 46 41 54 20 20 20] @ 0x03
#pragma description Extensible File Allocation Table (exFAT)
#pragma author Khoo Hao Yit
import std.mem;
import std.string;
import std.core;
#pragma eval_depth 0
struct BootSector {
std::mem::Bytes<3> jumpBoot;
char fileSystemName[8];
std::mem::Bytes<53> mustBeZero;
u64 partitionOffset;
u64 volumeLength;
u32 fatOffset;
u32 fatLength;
u32 clusterHeapOffset;
u32 clusterCount;
u32 firstClusterOfRootDirectory;
u32 volumeSerialNumber;
u16 fileSystemRevision;
u16 volumeFlags;
u8 bytesPerSectorShift;
u8 sectorsPerClusterShift;
u8 numberOfFats;
u8 driveSelect;
u8 percentInUse;
std::mem::Bytes<7> reserved;
std::mem::Bytes<390> bootCode;
std::mem::Bytes<2> bootSignature;
};
struct ExtendedBootSector {
std::mem::Bytes<bytesPerSector - 4> bootCode;
std::mem::Bytes<4> bootSignature;
};
struct Parameter {
std::mem::Bytes<16> guid;
std::mem::Bytes<32> data;
};
struct OemParameter {
Parameter parameters[10];
std::mem::Bytes<bytesPerSector - 480> reserved;
};
u64 bytesPerSector;
u64 startOfClusterHeap;
u64 bytesPerCluster;
u64 fatAddress;
struct BootRegion {
BootSector bootSector;
bytesPerSector = 1 << bootSector.bytesPerSectorShift;
bytesPerCluster = bytesPerSector << bootSector.sectorsPerClusterShift;
ExtendedBootSector extendedBootSectors[8];
OemParameter oemParameter;
std::mem::Bytes<bytesPerSector> reservedSector;
std::mem::Bytes<bytesPerSector> bootChecksum;
};
struct VolumeLabelEntry {
u8 entryType;
u8 characterCount;
char volumeLabel[22];
std::mem::Bytes<8> reserved;
};
bitfield FileAttribute {
readOnly : 1;
hidden : 1;
system : 1;
reserved : 1;
directory : 1;
archive : 1;
reserved1 : 10;
};
auto currentClusterIndex = -1;
auto currentClusterSize = -1;
auto currentIsDirectory = -1;
struct FileEntry {
u8 entryType;
u8 secondayEntryCount;
u16 checksum;
FileAttribute attributes;
std::mem::Bytes<2> reserved;
u32 creationDatetime;
u32 modificationDatetime;
u32 accessDatetime;
u8 creationTimeHundredths;
u8 modificationTimeHundredths;
u8 creationUtcOffset;
u8 modificationUtcOffset;
u8 accessUtcOffset;
std::mem::Bytes<7> reserved1;
currentIsDirectory = attributes.directory;
};
bitfield GeneralSecondaryFlags {
allocationPossible : 1;
noFatChain : 1;
customDefined : 6;
};
struct FileNameEntry {
u8 entryType;
u8 flags;
char16 name[15];
};
fn divideCeil(auto a, auto b) {
auto result = a / b;
auto remain = a % b;
if (remain) {
result += 1;
}
return result;
};
struct ContinuousFatRange<auto ClusterIndex, auto ClusterSize> {
u32 fatRange[ClusterSize] @ fatAddress + ClusterIndex * 4 [[sealed]];
} [[name(std::format("ContinuousFatRange<{}, {}>", ClusterIndex, ClusterSize))]];
struct FatChain {
auto clusterIndex = currentClusterIndex;
auto clusterSize = currentClusterSize;
currentClusterIndex = -1;
currentClusterSize = -1;
if (clusterIndex == -1) {
std::error(std::format("Invalid cluster index: {}", clusterIndex));
}
if (clusterSize == -1) {
auto startingFatRange = fatAddress + clusterIndex * 4;
u32 clusters[while(
$ == startingFatRange
|| $ == fatAddress + std::mem::read_unsigned($ - 4, 4) * 4
)] @ startingFatRange [[hidden]];
clusterSize = std::core::member_count(clusters);
}
ContinuousFatRange<clusterIndex, clusterSize> data @ 0;
try {
auto nextClusterIndex = clusters[clusterSize - 1];
if (nextClusterIndex != 0xffffffff && nextClusterIndex != 0) {
currentClusterIndex = nextClusterIndex;
FatChain next @ 0 [[inline]];
}
}
} [[name(std::format("FatChain<{}>", clusterIndex))]];
struct ContinuousDataRange<auto ClusterIndex, auto ClusterSize> {
std::mem::Bytes<ClusterSize * bytesPerCluster> dataRange @
startOfClusterHeap + ClusterIndex * bytesPerCluster;
} [[name(std::format("ContinuousDataRange<{}, {}>", ClusterIndex, ClusterSize))]];
struct DataChain {
auto clusterIndex = currentClusterIndex;
auto clusterSize = currentClusterSize;
currentClusterIndex = -1;
currentClusterSize = -1;
if (clusterIndex == -1) {
std::error(std::format("Invalid cluster index: {}", clusterIndex));
}
if (clusterSize == -1) {
auto startingFatRange = fatAddress + clusterIndex * 4;
u32 clusters[while(
$ == startingFatRange
|| $ == fatAddress + std::mem::read_unsigned($ - 4, 4) * 4
)] @ startingFatRange [[hidden]];
clusterSize = std::core::member_count(clusters);
}
ContinuousDataRange<clusterIndex, clusterSize> data @ 0;
try {
auto nextClusterIndex = clusters[clusterSize - 1];
if (nextClusterIndex != 0xffffffff && nextClusterIndex != 0) {
currentClusterIndex = nextClusterIndex;
DataChain next @ 0 [[inline]];
}
}
} [[name(std::format("DataChain<{}>", clusterIndex))]];
struct UpcaseTableEntry {
u8 entryType;
std::mem::Bytes<3> reserved;
u32 tableChecksum;
std::mem::Bytes<12> reserved1;
u32 firstCluster;
u64 dataLength;
ContinuousFatRange<firstCluster, 1> fatRange;
ContinuousDataRange<firstCluster, 1> dataRange;
};
struct AllocationBitmapEntry {
u8 entryType;
u8 bitmapFlags;
std::mem::Bytes<18> reserved;
u32 firstCluster;
u64 dataLength;
ContinuousFatRange<firstCluster, 1> fatRange;
ContinuousDataRange<firstCluster, 1> dataRange;
};
using StreamExtensionEntry;
struct DirectoryEntry {
u8 entryType @ addressof(this) [[hidden]];
match (entryType) {
(0x83 | 0x03): VolumeLabelEntry;
(0x81): AllocationBitmapEntry;
(0x82): UpcaseTableEntry;
(0x85 | 0x05): FileEntry;
(0xc0 | 0x40): StreamExtensionEntry;
(0xc1 | 0x41): FileNameEntry;
(_): std::mem::Bytes<32> [[name(std::format("UnknownEntry @ {:#X}", addressof(this)))]];
}
} [[inline]];
struct ContinuousDirectoryEntry<auto ClusterIndex, auto ClusterSize> {
DirectoryEntry entry[ClusterSize * bytesPerCluster / 32] @
startOfClusterHeap + ClusterIndex * bytesPerCluster
[[inline]];
} [[name(std::format("ContinuousDirectoryEntry<{}, {}>", ClusterIndex, ClusterSize))]];
struct DirectoryEntryChain {
auto clusterIndex = currentClusterIndex;
auto clusterSize = currentClusterSize;
currentClusterIndex = -1;
currentClusterSize = -1;
if (clusterIndex == -1) {
std::error(std::format("Invalid cluster index: {}", clusterIndex));
}
if (clusterSize == -1) {
auto startingFatRange = fatAddress + clusterIndex * 4;
u32 clusters[while(
$ == startingFatRange
|| $ == fatAddress + std::mem::read_unsigned($ - 4, 4) * 4
)] @ startingFatRange [[hidden]];
clusterSize = std::core::member_count(clusters);
}
ContinuousDirectoryEntry<clusterIndex, clusterSize> data @ 0;
try {
auto nextClusterIndex = clusters[clusterSize - 1];
if (nextClusterIndex != 0xffffffff && nextClusterIndex != 0) {
currentClusterIndex = nextClusterIndex;
DirectoryEntryChain next @ 0 [[inline]];
}
}
currentIsDirectory = -1;
} [[name(std::format("DirectoryEntryChain<{}>", clusterIndex))]];
struct StreamExtensionEntry {
u8 entryType;
GeneralSecondaryFlags secondaryFlags;
std::mem::Bytes<1> reserved;
u8 nameLength;
u16 nameHash;
std::mem::Bytes<2> reserved1;
u64 validDateLength;
std::mem::Bytes<4> reserved2;
u32 firstCluster;
u64 dataLength;
if (entryType & 0x80 && currentIsDirectory == 1) {
currentClusterIndex = firstCluster;
if (secondaryFlags.noFatChain) {
currentClusterSize = divideCeil(dataLength, bytesPerCluster);
}
FatChain fatChain;
currentClusterIndex = firstCluster;
if (secondaryFlags.noFatChain) {
currentClusterSize = divideCeil(dataLength, bytesPerCluster);
}
DirectoryEntryChain directoryChain;
}
else if (dataLength) {
currentClusterIndex = firstCluster;
if (secondaryFlags.noFatChain) {
currentClusterSize = divideCeil(dataLength, bytesPerCluster);
}
FatChain fatChain;
currentClusterIndex = firstCluster;
if (secondaryFlags.noFatChain) {
currentClusterSize = divideCeil(dataLength, bytesPerCluster);
}
DataChain data @ 0;
}
};
BootRegion bootRegion @ 0 [[name("BootRegion")]];
BootRegion backupBootRegion @ 12 * bytesPerSector [[name("BackupBootRegion")]];
startOfClusterHeap =
bootRegion.bootSector.clusterHeapOffset * bytesPerSector
- 2 * bytesPerCluster;
fatAddress =
bootRegion.bootSector.fatOffset * bytesPerSector;
std::mem::Bytes<bootRegion.bootSector.fatLength * bytesPerSector> fat @ fatAddress [[hidden]];
currentClusterIndex = bootRegion.bootSector.firstClusterOfRootDirectory;
FatChain rootDirectoryFatChain @ 0;
currentClusterIndex = bootRegion.bootSector.firstClusterOfRootDirectory;
DirectoryEntryChain rootDirectory @ 0;

108
patterns/fs/fat32.hexpat Normal file
View File

@@ -0,0 +1,108 @@
import std.core;
u64 bytesPerCluster;
struct FSInfo {
u32 leadSignature;
padding[480];
u32 structSignature;
u32 freeClusterCount;
u32 nextFreeCluster;
padding[12];
u32 trailSignature;
};
bitfield SequenceNumber {
padding : 1;
lastLogical : 1;
padding : 1;
number : 5;
} [[bitfield_order(std::core::BitfieldOrder::MostToLeastSignificant, 8)]];
enum EntryStatus : u8 {
Regular = 0x00,
DotEntry = 0x2E,
DeletedEntry = 0xE5
};
union EntryStatusOrSequenceNumber {
EntryStatus entryStatus;
SequenceNumber sequenceNumber;
};
bitfield Attributes {
readOnly : 1;
hidden : 1;
systemFile : 1;
volumeLabel : 1;
subdirectory : 1;
archive : 1;
padding : 2;
} [[bitfield_order(std::core::BitfieldOrder::LeastToMostSignificant, 8)]];
struct DirEntry {
char fileName[8];
char extension[3];
Attributes attributes;
u8 reserved[10];
u16 time, date;
u16 startingCluster;
u32 fileSize;
u8 data[fileSize] @ startingCluster * bytesPerCluster;
};
struct VFATDirEntry {
EntryStatusOrSequenceNumber entryStatusOrSequenceNumber;
char16 name1[5];
Attributes attributes;
u8 type;
u8 nameChecksum;
char16 name2[6];
u16 startingCluster;
char16 name3[2];
if (entryStatusOrSequenceNumber.sequenceNumber.number > 1)
VFATDirEntry nextLogicalEntry;
else
DirEntry physicalEntry;
};
struct FAT32 {
u8 jmpCode[3];
char oemName[8];
u16 bytesPerSector;
u8 sectorsPerCluster;
u16 reservedAreaSize;
u8 numFats;
u16 rootEntryCount;
u16 numSectors;
u8 mediaType;
u16 fatSize;
u16 sectorsPerTrack;
u16 numHeads;
u32 numHiddenSectors;
u32 numFsSectors;
u32 numFatSectors;
u16 extFlags;
u16 fsVersion;
u32 rootCluster;
u16 fsInfoSector;
u16 backupBootSector;
padding[12];
u8 driveNumber;
padding[1];
u8 bootSignature;
u32 volumeID;
char volumeLabel[11];
char fsType[8];
u8 bootstrapCode[420];
u16 signature;
bytesPerCluster = (sectorsPerCluster * 1024) * bytesPerSector;
FSInfo fsInfo @ addressof(this) + fsInfoSector * bytesPerSector;
VFATDirEntry rootDirEntry @ addressof(this) + rootCluster * bytesPerCluster;
};
FAT32 fat32 @ 0x00;

997
patterns/fs/ntfs.hexpat Normal file
View File

@@ -0,0 +1,997 @@
#pragma author Hrant Tadevosyan (Axcient, now ConnectWise)
#pragma description NT File System (NTFS)
#pragma endian little
// refs:
// - https://github.com/libyal/libfsntfs
// - https://github.com/tuxera/ntfs-3g
import std.core;
import std.array;
import std.ptr;
import std.io;
import std.mem;
import type.magic;
using ntfschar = u16;
using leVCN = u64;
using leLCN = u64;
using leLSN = u64;
using leMFT_REF = u64;
struct GUID {
u32 Data1;
u16 Data2;
u16 Data3;
u8 Data4[8];
} [[static]];
bitfield RUNLIST_HEADER {
unsigned offset : 4;
unsigned length : 4;
} [[bitfield_order(std::core::BitfieldOrder::MostToLeastSignificant, 8)]];
struct RUNLIST {
RUNLIST_HEADER header;
u8 count[header.length];
u8 lcn[header.offset];
if (header.length == 0)
break;
};
struct BIOS_PARAMETER_BLOCK {
u16 bytes_per_sector;
u8 sectors_per_cluster;
u16 reserved_sectors;
u8 fats;
u16 root_entries;
u16 sectors;
u8 media_type;
u16 sectors_per_fat;
u16 sectors_per_track;
u16 heads;
u32 hidden_sectors;
u32 large_sectors;
};
enum NTFS_RECORD_TYPES : u32 {
magic_FILE = 0x454c4946,
magic_INDX = 0x58444e49,
magic_HOLE = 0x454c4f48,
magic_RSTR = 0x52545352,
magic_RCRD = 0x44524352,
magic_CHKD = 0x444b4843,
magic_BAAD = 0x44414142,
magic_empty = 0xffffffff,
};
struct NTFS_RECORD {
NTFS_RECORD_TYPES magic;
u16 usa_ofs;
u16 usa_count;
};
enum MFT_RECORD_FLAGS : u16 {
MFT_RECORD_IN_USE = 0x0001,
MFT_RECORD_IS_DIRECTORY = 0x0002,
MFT_RECORD_IS_4 = 0x0004,
MFT_RECORD_IS_VIEW_INDEX = 0x0008,
MFT_REC_SPACE_FILLER = 0xffff,
};
enum ATTR_TYPES : u32 {
AT_UNUSED = 0,
AT_STANDARD_INFORMATION = 0x10,
AT_ATTRIBUTE_LIST = 0x20,
AT_FILE_NAME = 0x30,
AT_OBJECT_ID = 0x40,
AT_SECURITY_DESCRIPTOR = 0x50,
AT_VOLUME_NAME = 0x60,
AT_VOLUME_INFORMATION = 0x70,
AT_DATA = 0x80,
AT_INDEX_ROOT = 0x90,
AT_INDEX_ALLOCATION = 0xa0,
AT_BITMAP = 0xb0,
AT_REPARSE_POINT = 0xc0,
AT_EA_INFORMATION = 0xd0,
AT_EA = 0xe0,
AT_PROPERTY_SET = 0xf0,
AT_LOGGED_UTILITY_STREAM = 0x100,
AT_FIRST_USER_DEFINED_ATTRIBUTE = 0x1000,
AT_END = 0xffffffff,
};
enum ATTR_DEF_FLAGS : u32 {
ATTR_DEF_INDEXABLE = 0x02,
ATTR_DEF_MULTIPLE = 0x04,
ATTR_DEF_NOT_ZERO = 0x08,
ATTR_DEF_INDEXED_UNIQUE = 0x10,
ATTR_DEF_NAMED_UNIQUE = 0x20,
ATTR_DEF_RESIDENT = 0x40,
ATTR_DEF_ALWAYS_LOG = 0x80,
};
enum COLLATION_RULES : u32 {
COLLATION_BINARY = 0,
COLLATION_FILE_NAME = 1,
COLLATION_UNICODE_STRING = 2,
COLLATION_NTOFS_ULONG = 16,
COLLATION_NTOFS_SID = 17,
COLLATION_NTOFS_SECURITY_HASH = 18,
COLLATION_NTOFS_ULONGS = 19,
};
struct ATTR_DEF {
ntfschar name[0x40];
ATTR_TYPES type;
u32 display_rule;
COLLATION_RULES collation_rule;
ATTR_DEF_FLAGS flags;
u64 min_size;
u64 max_size;
};
enum ATTR_FLAGS : u16 {
ATTR_IS_COMPRESSED = 0x0001,
ATTR_COMPRESSION_MASK = 0x00ff,
ATTR_IS_ENCRYPTED = 0x4000,
ATTR_IS_SPARSE = 0x8000,
};
enum RESIDENT_ATTR_FLAGS : u8 {
RESIDENT_ATTR_IS_INDEXED = 0x01
};
enum FILE_ATTR_FLAGS : u32 {
FILE_ATTR_READONLY = 0x00000001,
FILE_ATTR_HIDDEN = 0x00000002,
FILE_ATTR_SYSTEM = 0x00000004,
FILE_ATTR_DIRECTORY = 0x00000010,
FILE_ATTR_ARCHIVE = 0x00000020,
FILE_ATTR_DEVICE = 0x00000040,
FILE_ATTR_NORMAL = 0x00000080,
FILE_ATTR_TEMPORARY = 0x00000100,
FILE_ATTR_SPARSE_FILE = 0x00000200,
FILE_ATTR_REPARSE_POINT = 0x00000400,
FILE_ATTR_COMPRESSED = 0x00000800,
FILE_ATTR_OFFLINE = 0x00001000,
FILE_ATTR_NOT_CONTENT_INDEXED = 0x00002000,
FILE_ATTR_ENCRYPTED = 0x00004000,
FILE_ATTRIBUTE_RECALL_ON_OPEN = 0x00040000,
FILE_ATTR_VALID_FLAGS = 0x00047fb7,
FILE_ATTR_VALID_SET_FLAGS = 0x000031a7,
FILE_ATTR_I30_INDEX_PRESENT = 0x10000000,
FILE_ATTR_VIEW_INDEX_PRESENT = 0x20000000,
};
enum FILE_NAME_TYPE_FLAGS : u8 {
FILE_NAME_POSIX = 0x00,
FILE_NAME_WIN32 = 0x01,
FILE_NAME_DOS = 0x02,
FILE_NAME_WIN32_AND_DOS = 0x03,
};
struct STANDARD_INFORMATION_HEADER {
u64 creation_time;
u64 last_data_change_time;
u64 last_mft_change_time;
u64 last_access_time;
FILE_ATTR_FLAGS file_attributes;
} [[static]];
struct STANDARD_INFORMATION_OLD {
STANDARD_INFORMATION_HEADER header [[inline]];
u8 reserved12[12];
} [[static]];
struct STANDARD_INFORMATION {
STANDARD_INFORMATION_HEADER header [[inline]];
u32 maximum_versions;
u32 version_number;
u32 class_id;
u32 owner_id;
u32 security_id;
u64 quota_charged;
u64 usn;
} [[static]];
struct FILE_NAME_ATTR_PACKED {
u16 packed_ea_size;
u16 reserved;
} [[static]];
union FILE_NAME_ATTR_FORM {
FILE_NAME_ATTR_PACKED packed [[inline]];
u32 reparse_point_tag;
} [[static]];
struct FILE_NAME_ATTR {
leMFT_REF parent_directory;
u64 creation_time;
u64 last_data_change_time;
u64 last_mft_change_time;
u64 last_access_time;
u64 allocated_size;
u64 data_size;
FILE_ATTR_FLAGS file_attributes;
FILE_NAME_ATTR_FORM form [[inline]];
u8 file_name_length;
FILE_NAME_TYPE_FLAGS file_name_type;
ntfschar file_name[file_name_length];
};
struct OBJECT_ID_ATTR_INFO {
GUID birth_volume_id;
GUID birth_object_id;
GUID domain_id;
};
union OBJECT_ID_ATTR_FORM {
OBJECT_ID_ATTR_INFO info [[inline]];
u8 extended_info[48];
};
struct OBJECT_ID_ATTR {
GUID object_id;
OBJECT_ID_ATTR_FORM form [[inline]];
};
enum VOLUME_FLAGS : u16 {
VOLUME_IS_DIRTY = 0x0001,
VOLUME_RESIZE_LOG_FILE = 0x0002,
VOLUME_UPGRADE_ON_MOUNT = 0x0004,
VOLUME_MOUNTED_ON_NT4 = 0x0008,
VOLUME_DELETE_USN_UNDERWAY = 0x0010,
VOLUME_REPAIR_OBJECT_ID = 0x0020,
VOLUME_CHKDSK_UNDERWAY = 0x4000,
VOLUME_MODIFIED_BY_CHKDSK = 0x8000,
VOLUME_FLAGS_MASK = 0xc03f,
};
struct VOLUME_INFORMATION {
u64 reserved;
u8 major_ver;
u8 minor_ver;
VOLUME_FLAGS flags;
} [[static]];
enum SECURITY_DESCRIPTOR_CONTROL : u16 {
SE_OWNER_DEFAULTED = 0x0001,
SE_GROUP_DEFAULTED = 0x0002,
SE_DACL_PRESENT = 0x0004,
SE_DACL_DEFAULTED = 0x0008,
SE_SACL_PRESENT = 0x0010,
SE_SACL_DEFAULTED = 0x0020,
SE_DACL_AUTO_INHERIT_REQ = 0x0100,
SE_SACL_AUTO_INHERIT_REQ = 0x0200,
SE_DACL_AUTO_INHERITED = 0x0400,
SE_SACL_AUTO_INHERITED = 0x0800,
SE_DACL_PROTECTED = 0x1000,
SE_SACL_PROTECTED = 0x2000,
SE_RM_CONTROL_VALID = 0x4000,
SE_SELF_RELATIVE = 0x8000,
};
enum ACE_TYPES : u8 {
ACCESS_MIN_MS_ACE_TYPE = 0,
ACCESS_ALLOWED_ACE_TYPE = 0,
ACCESS_DENIED_ACE_TYPE = 1,
SYSTEM_AUDIT_ACE_TYPE = 2,
SYSTEM_ALARM_ACE_TYPE = 3,
ACCESS_MAX_MS_V2_ACE_TYPE = 3,
ACCESS_ALLOWED_COMPOUND_ACE_TYPE = 4,
ACCESS_MAX_MS_V3_ACE_TYPE = 4,
ACCESS_MIN_MS_OBJECT_ACE_TYPE = 5,
ACCESS_ALLOWED_OBJECT_ACE_TYPE = 5,
ACCESS_DENIED_OBJECT_ACE_TYPE = 6,
SYSTEM_AUDIT_OBJECT_ACE_TYPE = 7,
SYSTEM_ALARM_OBJECT_ACE_TYPE = 8,
ACCESS_MAX_MS_OBJECT_ACE_TYPE = 8,
ACCESS_MAX_MS_V4_ACE_TYPE = 8,
ACCESS_MAX_MS_ACE_TYPE = 8,
ACCESS_ALLOWED_CALLBACK_ACE_TYPE = 9,
ACCESS_DENIED_CALLBACK_ACE_TYPE = 10,
ACCESS_ALLOWED_CALLBACK_OBJECT_ACE_TYPE = 11,
ACCESS_DENIED_CALLBACK_OBJECT_ACE_TYPE = 12,
SYSTEM_AUDIT_CALLBACK_ACE_TYPE = 13,
SYSTEM_ALARM_CALLBACK_ACE_TYPE = 14,
SYSTEM_AUDIT_CALLBACK_OBJECT_ACE_TYPE = 15,
SYSTEM_ALARM_CALLBACK_OBJECT_ACE_TYPE = 16,
SYSTEM_MANDATORY_LABEL_ACE_TYPE = 17,
SYSTEM_RESOURCE_ATTRIBUTE_ACE_TYPE = 18,
SYSTEM_SCOPED_POLICY_ID_ACE_TYPE = 19,
SYSTEM_PROCESS_TRUST_LABEL_ACE_TYPE = 20,
};
enum ACE_FLAGS : u8 {
OBJECT_INHERIT_ACE = 0x01,
CONTAINER_INHERIT_ACE = 0x02,
NO_PROPAGATE_INHERIT_ACE = 0x04,
INHERIT_ONLY_ACE = 0x08,
INHERITED_ACE = 0x10,
VALID_INHERIT_FLAGS = 0x1f,
SUCCESSFUL_ACCESS_ACE_FLAG = 0x40,
FAILED_ACCESS_ACE_FLAG = 0x80,
};
struct ACE_HEADER {
ACE_TYPES type;
ACE_FLAGS flags;
u16 size;
};
struct SID {
u8 revision;
u8 sub_authority_count;
u8 identifier_authority[6];
u32 sub_authority[sub_authority_count];
};
enum ACCESS_MASK : u32 {
FILE_READ_DATA = 0x00000001,
FILE_LIST_DIRECTORY = 0x00000001,
FILE_WRITE_DATA = 0x00000002,
FILE_ADD_FILE = 0x00000002,
FILE_APPEND_DATA = 0x00000004,
FILE_ADD_SUBDIRECTORY = 0x00000004,
FILE_READ_EA = 0x00000008,
FILE_WRITE_EA = 0x00000010,
FILE_EXECUTE = 0x00000020,
FILE_TRAVERSE = 0x00000020,
FILE_DELETE_CHILD = 0x00000040,
FILE_READ_ATTRIBUTES = 0x00000080,
FILE_WRITE_ATTRIBUTES = 0x00000100,
DELETE = 0x00010000,
READ_CONTROL = 0x00020000,
WRITE_DAC = 0x00040000,
WRITE_OWNER = 0x00080000,
SYNCHRONIZE = 0x00100000,
STANDARD_RIGHTS_READ = 0x00020000,
STANDARD_RIGHTS_WRITE = 0x00020000,
STANDARD_RIGHTS_EXECUTE = 0x00020000,
STANDARD_RIGHTS_REQUIRED = 0x000f0000,
STANDARD_RIGHTS_ALL = 0x001f0000,
ACCESS_SYSTEM_SECURITY = 0x01000000,
MAXIMUM_ALLOWED = 0x02000000,
GENERIC_ALL = 0x10000000,
GENERIC_EXECUTE = 0x20000000,
GENERIC_WRITE = 0x40000000,
GENERIC_READ = 0x80000000,
};
struct ACCESS_ALLOWED_ACE {
ACE_HEADER header [[inline]];
ACCESS_MASK mask;
SID sid;
};
using ACCESS_DENIED_ACE = ACCESS_ALLOWED_ACE;
using SYSTEM_ALARM_ACE = ACCESS_ALLOWED_ACE;
using SYSTEM_AUDIT_ACE = ACCESS_ALLOWED_ACE;
enum OBJECT_ACE_FLAGS : u32 {
ACE_OBJECT_TYPE_PRESENT = 1,
ACE_INHERITED_OBJECT_TYPE_PRESENT = 2,
};
struct ACCESS_ALLOWED_OBJECT_ACE {
ACE_HEADER header [[inline]];
ACCESS_MASK mask;
OBJECT_ACE_FLAGS object_flags;
GUID object_type;
GUID inherited_object_type;
SID sid;
};
using ACCESS_DENIED_OBJECT_ACE = ACCESS_ALLOWED_OBJECT_ACE;
using SYSTEM_AUDIT_OBJECT_ACE = ACCESS_ALLOWED_OBJECT_ACE;
using SYSTEM_ALARM_OBJECT_ACE = ACCESS_ALLOWED_OBJECT_ACE;
struct ACL {
u8 revision;
u8 alignment1;
u16 size;
u16 ace_count;
u16 alignment2;
std::mem::AlignTo<4>;
padding[size];
std::mem::AlignTo<4>;
};
#define SECURITY_DESCRIPTOR_RELATIVE_SIZE (20)
struct SECURITY_DESCRIPTOR_RELATIVE {
u32 struct_start = $;
u8 revision;
u8 alignment;
u16 control; // SECURITY_DESCRIPTOR_CONTROL
u32 owner;
u32 group;
u32 sacl;
u32 dacl;
if (owner > 0) {
u32 owner_bytes = $ - struct_start;
if (owner > owner_bytes) {
padding[owner - owner_bytes];
}
SID owner_sid;
}
if (group > 0) {
u32 group_bytes = $ - struct_start;
if (group > group_bytes) {
padding[group - group_bytes];
}
SID group_sid;
}
if (control & SECURITY_DESCRIPTOR_CONTROL::SE_SACL_PRESENT) {
u32 sacl_bytes = $ - struct_start;
if (sacl > sacl_bytes) {
padding[sacl - sacl_bytes];
ACL acl;
}
}
if (control & SECURITY_DESCRIPTOR_CONTROL::SE_DACL_PRESENT) {
u32 dacl_bytes = $ - struct_start;
if (dacl > dacl_bytes) {
padding[dacl - dacl_bytes];
ACL acl;
}
}
};
enum INDEX_HEADER_FLAGS : u8 {
SMALL_INDEX = 0,
LARGE_INDEX = 1,
LEAF_NODE = 0,
INDEX_NODE = 1,
NODE_MASK = 1,
};
struct INDEX_HEADER {
u32 entries_offset;
u32 index_length;
u32 allocated_size;
INDEX_HEADER_FLAGS ih_flags;
u8 reserved[3];
};
struct REPARSE_INDEX_KEY {
u32 reparse_tag;
leMFT_REF file_id;
};
struct SDS_ENTRY {
u32 hash;
u32 security_id;
u64 offset;
u32 length;
SECURITY_DESCRIPTOR_RELATIVE sid;
};
using SII_INDEX_KEY = u32 ;
struct SDH_INDEX_KEY {
u32 hash;
u32 security_id;
};
enum INDEX_ENTRY_FLAGS : u16 {
INDEX_ENTRY_NODE = 1,
INDEX_ENTRY_END = 2,
INDEX_ENTRY_SPACE_FILLER = 0xffff,
};
struct INDEX_ENTR_HEADER_PTR {
u16 data_offset;
u16 data_length;
u32 reservedV;
};
union INDEX_ENTR_HEADER_REF {
leMFT_REF indexed_file;
INDEX_ENTR_HEADER_PTR ptr;
};
struct INDEX_ENTRY_HEADER {
INDEX_ENTR_HEADER_REF file_ref;
u16 length;
u16 key_length;
u16 flags; // INDEX_ENTRY_FLAGS
u16 reserved;
};
struct INDEX_ENTRY_FILE_NAME {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
FILE_NAME_ATTR key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
struct INDEX_ENTRY_SII {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
SII_INDEX_KEY key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
struct INDEX_ENTRY_SDH {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
SDH_INDEX_KEY key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
struct INDEX_ENTRY_OBJ_ID {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
GUID key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
struct INDEX_ENTRY_REPARSE {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
REPARSE_INDEX_KEY key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
struct INDEX_ENTRY_SID {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
SID key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
struct INDEX_ENTRY_OWNER_ID {
INDEX_ENTRY_HEADER header;
if (!(header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END)) {
u64 key;
}
if (header.flags & INDEX_ENTRY_FLAGS::INDEX_ENTRY_NODE) {
leVCN vcn;
}
std::mem::AlignTo<8>;
};
fn index_get_flags(u64 offset) {
INDEX_ENTRY_HEADER index_entry_header @ offset;
return index_entry_header.flags;
};
struct INDEX_BLOCK {
NTFS_RECORD header [[inline]];
leLSN lsn;
leVCN index_block_vcn;
u32 index_head = $;
INDEX_HEADER index;
if (index.entries_offset > sizeof (index)) {
padding[index.entries_offset - sizeof (index)];
}
INDEX_ENTRY_FILE_NAME ents[while(!(index_get_flags($) & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END))];
INDEX_ENTRY_FILE_NAME ent_end;
u32 index_used = $ - index_head;
if (index.index_length > index_used) {
padding[index.index_length - index_used];
}
};
struct INDEX_ROOT {
ATTR_TYPES type;
COLLATION_RULES collation_rule;
u32 index_block_size;
s8 clusters_per_index_block;
u8 reserved[3];
u32 index_head = $;
INDEX_HEADER index;
if (index.entries_offset > sizeof (index)) {
padding[index.entries_offset - sizeof (index)];
}
match (type) {
(ATTR_TYPES::AT_FILE_NAME): {
INDEX_ENTRY_FILE_NAME ents[while(!(index_get_flags($) & INDEX_ENTRY_FLAGS::INDEX_ENTRY_END))];
INDEX_ENTRY_FILE_NAME ent_end;
}
}
u32 index_used = $ - index_head;
if (index.index_length > index_used) {
padding[index.index_length - index_used];
}
};
struct RESTART_PAGE_HEADER {
NTFS_RECORD_TYPES magic;
u16 usa_ofs;
u16 usa_count;
leLSN chkdsk_lsn;
u32 system_page_size;
u32 log_page_size;
u16 restart_area_offset;
u16 minor_ver;
u16 major_ver;
u16 usn;
};
enum RESTART_AREA_FLAGS : u16 {
RESTART_VOLUME_IS_CLEAN = 0x0002,
RESTART_SPACE_FILLER = 0xffff,
};
struct RESTART_AREA {
leLSN current_lsn;
u16 log_clients;
u16 client_free_list;
u16 client_in_use_list;
RESTART_AREA_FLAGS flags;
u32 seq_number_bits;
u16 restart_area_length;
u16 client_array_offset;
u64 file_size;
u32 last_lsn_data_length;
u16 log_record_header_length;
u16 log_page_data_offset;
u32 restart_log_open_count;
u32 reserved;
};
struct LOG_CLIENT_RECORD {
leLSN oldest_lsn;
leLSN client_restart_lsn;
u16 prev_client;
u16 next_client;
u16 seq_number;
u8 reserved[6];
u32 client_name_length;
ntfschar client_name[64];
};
#define ATTR_RECORD_HEADER_SIZE (16)
#define ATTR_RECORD_RESIDENT_SIZE (16 + 8)
#define ATTR_RECORD_NONRESIDENT_SIZE (16 + 48)
#define ATTR_RECORD_NONRESIDENT_CMPR_SIZE (16 + 48 + 8)
struct ATTR_RECORD {
ATTR_TYPES type;
u32 length;
u8 non_resident;
u8 name_length;
u16 name_offset;
u16 flags; // ATTR_FLAGS
u16 instance;
u32 name_offset_delta = 0;
if (!non_resident) {
u32 value_length;
u16 value_offset;
RESIDENT_ATTR_FLAGS resident_flags;
s8 reservedR;
if (name_offset > ATTR_RECORD_RESIDENT_SIZE) {
name_offset_delta = name_offset - ATTR_RECORD_RESIDENT_SIZE;
padding[name_offset_delta];
}
} else {
leVCN lowest_vcn;
leVCN highest_vcn;
u16 mapping_pairs_offset;
u8 compression_unit;
u8 reserved1[5];
u64 allocated_size;
u64 data_size;
u64 initialized_size;
if (flags & ATTR_FLAGS::ATTR_IS_COMPRESSED) {
u64 compressed_size;
if (name_offset > ATTR_RECORD_NONRESIDENT_CMPR_SIZE) {
name_offset_delta = name_offset - ATTR_RECORD_NONRESIDENT_CMPR_SIZE;
padding[name_offset_delta];
}
} else {
if (name_offset > ATTR_RECORD_NONRESIDENT_SIZE) {
name_offset_delta = name_offset - ATTR_RECORD_NONRESIDENT_SIZE;
padding[name_offset_delta];
}
}
}
u32 name_length_bytes = name_length * sizeof (ntfschar);
if (name_length > 0) {
ntfschar name[name_length];
}
if (non_resident) {
if (flags & ATTR_FLAGS::ATTR_IS_COMPRESSED) {
if (mapping_pairs_offset > (ATTR_RECORD_NONRESIDENT_CMPR_SIZE + name_offset_delta + name_length_bytes)) {
padding[mapping_pairs_offset - (ATTR_RECORD_NONRESIDENT_CMPR_SIZE + name_length_bytes)];
}
} else {
if (mapping_pairs_offset > (ATTR_RECORD_NONRESIDENT_SIZE + name_offset_delta + name_length_bytes)) {
padding[mapping_pairs_offset - (ATTR_RECORD_NONRESIDENT_SIZE + name_length_bytes)];
}
}
RUNLIST runlist[while(true)];
} else {
u32 value_offset_delta = 0;
if (value_offset > (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes)) {
value_offset_delta = value_offset - (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes);
padding[value_offset_delta];
}
match (type) {
(ATTR_TYPES::AT_UNUSED): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_STANDARD_INFORMATION): {
if (value_length > sizeof (STANDARD_INFORMATION_OLD)) {
STANDARD_INFORMATION info;
} else {
STANDARD_INFORMATION_OLD info;
}
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta + sizeof (info));
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_ATTRIBUTE_LIST): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_FILE_NAME): FILE_NAME_ATTR file;
(ATTR_TYPES::AT_OBJECT_ID): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_SECURITY_DESCRIPTOR): {
SECURITY_DESCRIPTOR_RELATIVE sec;
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta + sizeof (sec));
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_VOLUME_NAME): ntfschar vol_name[value_length / sizeof (ntfschar)];
(ATTR_TYPES::AT_VOLUME_INFORMATION): VOLUME_INFORMATION vol_info;
(ATTR_TYPES::AT_DATA): {
u8 buffer[value_length];
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta + sizeof (buffer));
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_INDEX_ROOT): {
INDEX_ROOT root;
}
(ATTR_TYPES::AT_INDEX_ALLOCATION): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_BITMAP): u8 buffer[value_length];
(ATTR_TYPES::AT_REPARSE_POINT): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_EA_INFORMATION): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_EA): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_PROPERTY_SET): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_LOGGED_UTILITY_STREAM): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_FIRST_USER_DEFINED_ATTRIBUTE): {
u32 attr_sum = (ATTR_RECORD_RESIDENT_SIZE + name_offset_delta + name_length_bytes + value_offset_delta);
if (length > attr_sum) {
padding[length - attr_sum];
}
}
(ATTR_TYPES::AT_END): {
// length is no longer valid
}
}
}
std::mem::AlignTo<8>;
};
fn attr_get_type(u64 offset) {
ATTR_RECORD attr @ offset;
return attr.type;
};
#define MFT_RECORD_SIZE (sizeof (NTFS_RECORD) + 40)
struct MFT_RECORD {
NTFS_RECORD header [[inline]];
leLSN lsn;
u16 sequence_number;
u16 link_count;
u16 attrs_offset;
MFT_RECORD_FLAGS flags;
u32 bytes_in_use;
u32 bytes_allocated;
leMFT_REF base_mft_record;
u16 next_attr_instance;
u16 reserved;
u32 mft_record_number;
if (header.usa_count > 0) {
padding[header.usa_ofs - MFT_RECORD_SIZE];
}
u16 update_sequence[header.usa_count];
std::mem::AlignTo<8>;
ATTR_RECORD attrs[while(attr_get_type($) != ATTR_TYPES::AT_END)];
ATTR_RECORD attr_end;
};
struct NTFS_BOOT_SECTOR {
u8 jump[3];
u64 oem_id;
BIOS_PARAMETER_BLOCK bpb;
u8 physical_drive;
u8 current_head;
u8 extended_boot_signature;
u8 reserved2;
u64 number_of_sectors;
u64 mft_lcn;
u64 mftmirr_lcn;
s8 clusters_per_mft_record;
u8 reserved0[3];
s8 clusters_per_index_record;
u8 reserved1[3];
u64 volume_serial_number;
u32 checksum;
u8 bootstrap[426];
u16 end_of_sector_marker;
};
fn mft_get_dat_attr_lcn(MFT_RECORD record, ATTR_TYPES type = ATTR_TYPES::AT_DATA) {
u64 lcn = 0;
for (u64 i = 0, i < record.next_attr_instance, i += 1) {
if (record.attrs[i].type == type) {
for (s64 j = record.attrs[i].runlist[0].header.offset - 1, j >= 0, j -= 1) {
lcn |= record.attrs[i].runlist[0].lcn[j] << (8 * j);
}
break;
}
}
return lcn;
};
// ============= VBR =============
NTFS_BOOT_SECTOR nbs @ 0x00 [[name("VBR")]];
u32 cluster_size = nbs.bpb.bytes_per_sector * nbs.bpb.sectors_per_cluster;
u32 mft_rec_size = 1 << -nbs.clusters_per_mft_record;
// ============= $MFT =============
MFT_RECORD mft @ nbs.mft_lcn * cluster_size + 0 * mft_rec_size [[name("$MFT")]];
MFT_RECORD mftmirr @ nbs.mftmirr_lcn * cluster_size [[name("$MFT Copy")]];
// ============= $MFTMirr =============
MFT_RECORD mft_mirr @ nbs.mft_lcn * cluster_size + 1 * mft_rec_size [[name("$MFTMirr")]];
// ============= $LogFile =============
MFT_RECORD mft_log @ nbs.mft_lcn * cluster_size + 2 * mft_rec_size [[name("$LogFile")]];
// ============= $Volume =============
MFT_RECORD mft_vol @ nbs.mft_lcn * cluster_size + 3 * mft_rec_size [[name("$Volume")]];
// ============= $AttrDef =============
MFT_RECORD mft_adef @ nbs.mft_lcn * cluster_size + 4 * mft_rec_size [[name("$AttrDef")]];
ATTR_DEF adef_dat_buf[16] @ mft_get_dat_attr_lcn(mft_adef) * cluster_size [[name("$AttrDef, array")]];
// ============= $I30 (Root) =============
MFT_RECORD mft_root @ nbs.mft_lcn * cluster_size + 5 * mft_rec_size [[name("$I30 (Root)")]];
INDEX_BLOCK root_index_block @ mft_get_dat_attr_lcn(mft_root, ATTR_TYPES::AT_INDEX_ALLOCATION) * cluster_size [[name("$I30 (Root), INDEX_BLOCK")]];
// ============= $Bitmap =============
MFT_RECORD mft_bm @ nbs.mft_lcn * cluster_size + 6 * mft_rec_size [[name("$Bitmap")]];
// ============= $Boot =============
MFT_RECORD mft_boot @ nbs.mft_lcn * cluster_size + 7 * mft_rec_size [[name("$Boot")]];
// ============= $BadClus =============
MFT_RECORD mft_badclus @ nbs.mft_lcn * cluster_size + 8 * mft_rec_size [[name("$BadClus")]];
// ============= $Secure ($Quota for NTFS 1.2) =============
MFT_RECORD mft_sec @ nbs.mft_lcn * cluster_size + 9 * mft_rec_size [[name("$Secure")]];
// ============= $UpCase =============
MFT_RECORD mft_uc @ nbs.mft_lcn * cluster_size + 10 * mft_rec_size [[name("$UpCase")]];
u8 uc_tbl[1] @ mft_get_dat_attr_lcn(mft_uc) * cluster_size [[name("$UpCase, tabl")]];
// ============= $Extend =============
MFT_RECORD mft_ext @ nbs.mft_lcn * cluster_size + 11 * mft_rec_size [[name("$Extend")]];

231
patterns/fs/pattern.hexpat Normal file
View File

@@ -0,0 +1,231 @@
#pragma author WerWolv
#pragma description Drive File System
#pragma MIME application/x-ima
import std.io;
import std.core;
import type.magic;
import type.guid;
import type.base;
import * from fs.fat32 as FAT32Partition;
import * from fs.exfat as ExFATPartition;
import * from fs.ntfs as NTFSPartition;
const u32 SectorSize = 512;
struct DiskTimeStamp {
u8 seconds, minutes, hours;
};
enum DiskProtection : u16 {
None = 0x0000,
CopyProtected = 0x5A5A
};
bitfield CHS {
h : 8;
s : 6;
c : 10;
} [[format("chs_formatter")]];
fn chs_formatter(CHS chs) {
return std::format("({:X}, {:X}, {:X}) | 0x{:X}", chs.c, chs.h, chs.s, (chs.c * 16 + chs.h) * 63 + (chs.s - 1));
};
enum PartitionStatus : u8 {
None = 0x00,
Active = 0x80
};
enum MBRPartitionType : u8 {
Empty = 0x00,
FAT12 = 0x01,
XENIXRoot = 0x02,
XENIXUsr = 0x03,
FAT16_16_32MB = 0x04,
ExtendedCHS = 0x05,
FAT16_32MBPlus = 0x06,
NTFS = 0x07,
AIX = 0x08,
AIXBootable = 0x09,
OS2BootManager = 0x0A,
FAT32_CHS = 0x0B,
FAT32_LBA = 0x0C,
FAT16_LBA = 0x0E,
ExtendedLBA = 0x0F,
OPUS = 0x10,
HiddenFAT12 = 0x11,
CompaqDiagnostics = 0x12,
HiddenFAT16_16_32MB = 0x14,
HiddenFAT16_32MBPlus = 0x16,
HiddenNTFS = 0x17,
ASTSmartSleep = 0x18,
HiddenFAT32_CHS = 0x1B,
HiddenFAT32_LBA = 0x1C,
HiddenFAT16_LBA = 0x1E,
NEC_DOS = 0x24,
WindowsRecovery = 0x27,
Plan9 = 0x39,
PowerQuest = 0x3C,
Venix286 = 0x40,
PPC_PReP_Boot = 0x41,
SFS = 0x42,
QNX4_x = 0x4D,
QNX4_x_2ndPart = 0x4E,
QNX4_x_3rdPart = 0x4F,
OnTrackDM = 0x50,
OnTrackDM6Aux1 = 0x51,
CP_M = 0x52,
OnTrackDM6Aux3 = 0x53,
OnTrackDM6 = 0x54,
EZDrive = 0x55,
GoldenBow = 0x56,
PriamEDisk = 0x5C,
SpeedStor = 0x61,
GNU_HURD = 0x63,
NovellNetware286 = 0x64,
NovellNetware386 = 0x65,
DiskSecureMultiBoot = 0x70,
PC_IX = 0x75,
XOSL = 0x78,
OldMinix = 0x80,
LinuxMinix = 0x81,
LinuxSwap = 0x82,
Linux = 0x83,
OS2HiddenCDrive = 0x84,
LinuxExtended = 0x85,
NTFSVolumeSet = 0x86,
NTFSVolumeSet2 = 0x87,
LinuxLVM = 0x8E,
Amoeba = 0x93,
AmoebaBBT = 0x94,
BSD_OS = 0x9F,
IBMThinkpadHibernation = 0xA0,
FreeBSD = 0xA5,
OpenBSD = 0xA6,
NeXTSTEP = 0xA7,
MacOSX = 0xA8,
NetBSD = 0xA9,
BSDIFS = 0xB7,
BSDISwap = 0xB8,
BootWizardHidden = 0xBB,
DRDOSFAT12 = 0xC1,
DRDOSFAT16 = 0xC4,
DRDOSFAT16B = 0xC6,
Syrinx = 0xC7,
NonFSData = 0xDA,
CP_M_CTOS = 0xDB,
DellUtility = 0xDE,
BootIt = 0xDF,
DOSAccess = 0xE1,
DOSRO = 0xE3,
SpeedStor2 = 0xE4,
BeOS = 0xEB,
GPTProtective = 0xEE,
EFI_System = 0xEF,
LinuxPA_RISC = 0xF0,
SpeedStor3 = 0xF1,
DOSSecondary = 0xF2,
LinuxRAID = 0xFD,
LANstep = 0xFE,
Unknown = 0xFF
};
enum GPTPartitionType : u128 {
UnusedEntry = u128("\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"),
MicrosoftReservedPartition = u128("\x16\xE3\xC9\xE3\x5C\x0B\xB8\x4D\x81\x7D\xF9\x2D\xF0\x02\x15\xAE"),
PartitionBasicData = u128("\xA2\xA0\xD0\xEB\xE5\xB9\x33\x44\x87\xC0\x68\xB6\xB7\x26\x99\xC7")
};
bitfield GPTAttributes {
bool platform_required : 1;
padding : 59;
bool read_only : 1;
bool shadow_copy : 1;
bool hidden : 1;
bool no_drive_letter : 1;
};
struct GPTEntry {
GPTPartitionType partitionType [[no_unique_address]];
type::GUID partitionTypeGuid;
type::GUID partitionGuid;
u64 firstLba;
u64 lastLba;
GPTAttributes attribute;
char16 partitionName[36];
padding[parent.entrySize - 128];
u64 partitionOffset = firstLba * SectorSize [[export]];
match (partitionType) {
(GPTPartitionType::UnusedEntry): {}
(GPTPartitionType::MicrosoftReservedPartition):
std::mem::Bytes<(lastLba - firstLba + 1) * SectorSize> microsoftReservedPartition @ partitionOffset;
(GPTPartitionType::PartitionBasicData): {}
(_): std::error(std::format("Unknown GPT Partition Type {}", partitionType));
}
};
struct GUIDPartitionTable {
type::Magic<"EFI PART"> signature;
u16 versionMinor;
u16 versionMajor;
u32 headerSize;
type::Hex<u32> headerCrc32;
u32 reserved;
u64 currentLba;
u64 backupLba;
u64 firstUsableLba;
u64 lastUsableLba;
type::GUID diskGuid;
u64 entryLba;
u32 entryCount;
u32 entrySize;
type::Hex<u32> entryCrc32;
GPTEntry entries[entryCount] @ addressof(this) + (entryLba - currentLba) * SectorSize;
};
struct PartitionEntry {
PartitionStatus status;
CHS chsFirstSectorAddress;
MBRPartitionType type;
CHS chsLastSectorAddress;
u32 lbaFirstSectorAddress;
u32 numSectors;
u64 partitionOffset = lbaFirstSectorAddress * SectorSize [[export]];
match (type) {
(MBRPartitionType::Empty): continue;
(MBRPartitionType::FAT32_LBA | MBRPartitionType::FAT32_CHS):
FAT32Partition fat32Partition @ partitionOffset;
(MBRPartitionType::NTFS): {
char magic[8] @ partitionOffset + 3;
if (magic == "NTFS ")
NTFSPartition ntfsPartition @ partitionOffset;
else if (magic == "EXFAT ")
ExFATPartition exfatPartiton @ partitionOffset;
else
std::error("Unknown MBR partition with NTFS Type ID");
}
(MBRPartitionType::GPTProtective):
GUIDPartitionTable gpt @ partitionOffset;
(_): std::error(std::format("Unknown MBR Partition Type {}", type));
}
};
struct MasterBootRecord {
u8 bootstrapCodeArea1[218];
padding[2];
u8 originalPhysicalDrive;
DiskTimeStamp diskTimeStamp;
u8 bootstrapCodeArea2[216];
u32 diskSignature;
DiskProtection diskProtection;
PartitionEntry partitionEntries[4];
u16 bootSignature;
};
MasterBootRecord mbr @ 0x00;

362
patterns/hprof.hexpat Normal file
View File

@@ -0,0 +1,362 @@
#pragma author WerWolv
#pragma description Java HPROF Profiler Data Format
#pragma endian big
#pragma magic [ "JAVA PROFILE" ] @ 0x00
#pragma array_limit 0
#pragma pattern_limit 0
import std.mem;
import std.io;
import std.sys;
import std.core;
enum Tag : u8 {
STRING_IN_UTF8 = 0x01,
LOAD_CLASS = 0x02,
UNLOAD_CLASS = 0x03,
STACK_FRAME = 0x04,
STACK_TRACE = 0x05,
ALLOC_SITES = 0x06,
HEAP_SUMMARY = 0x07,
START_THREAD = 0x0A,
END_THREAD = 0x0B,
HEAP_DUMP = 0x0C,
HEAP_DUMP_SEGMENT = 0x1C,
HEAP_DUMP_END = 0x2C,
CPU_SAMPLES = 0x0D,
CONTROL_SETTINGS = 0x0E,
};
u32 id_size = 0;
struct ID {
match (id_size) {
(1): u8 value;
(2): u16 value;
(4): u32 value;
(8): u64 value;
(_): std::error("Invalid ID size");
}
} [[sealed, format("format_id")]];
fn format_id(ref auto id) {
return std::format("0x{:0{}X}", id.value, id_size * 2);
};
struct StringInUTF8 {
ID id;
char string[parent.length - sizeof(id)];
};
struct LoadClass {
u32 classSerialNumber;
ID classObjectId;
u32 stackTraceSerialNumber;
ID classNameStringID;
};
struct StackFrame {
ID stackFrameId;
ID methodNameStringId;
ID methodSignatureStringId;
ID sourceFileNameStringId;
u32 classSerialNumber;
u32 lineInformation;
};
struct StackTrace {
u32 stackTraceSerialNumber;
u32 threadSerialNumber;
u32 numberOfFrames;
ID stackFrameIds[while(!std::mem::reached(addressof(this) + parent.length))];
};
enum SubTag : u8 {
RootUnknown = 0xFF,
RootJNIGlobal = 0x01,
RootJNILocal = 0x02,
RootJavaFrame = 0x03,
RootNativeStack = 0x04,
RootStickyClass = 0x05,
RootThreadBlock = 0x06,
RootMonitorUsed = 0x07,
RootThreadObject = 0x08,
ClassDump = 0x20,
InstanceDump = 0x21,
ObjectArrayDump = 0x22,
PrimitiveArrayDump = 0x23
};
enum EntryType : u8 {
Object = 2,
Boolean = 4,
Char = 5,
Float = 6,
Double = 7,
Byte = 8,
Short = 9,
Int = 10,
Long = 11
};
struct BasicType<auto Tag> {
match (Tag) {
(EntryType::Object): ID value;
(EntryType::Boolean): bool value;
(EntryType::Char): { padding[1]; char value; }
(EntryType::Float): float value;
(EntryType::Double): double value;
(EntryType::Byte): u8 value;
(EntryType::Short): u16 value;
(EntryType::Int): u32 value;
(EntryType::Long): u64 value;
(_): std::error("Invalid BasicType type");
}
} [[sealed, format("format_basic_type")]];
fn format_basic_type(ref auto basicType) {
return std::format("{}", basicType.value);
};
struct ConstantPoolEntry {
u16 constantPoolIndex;
EntryType type;
BasicType<type> value;
};
struct ConstantPoolArray {
u16 size;
ConstantPoolEntry entries[size];
};
struct StaticFieldEntry {
ID staticFieldNameStringId;
EntryType type;
BasicType<type> value;
};
struct StaicFieldArray {
u16 size;
StaticFieldEntry entries[size];
};
struct InstanceField {
ID fieldNameStringId;
EntryType fieldType;
};
struct InstanceFieldsArray {
u16 size;
InstanceField instanceFields[size];
};
struct HeapDump {
SubTag subTag;
match (subTag) {
(SubTag::RootUnknown): {
ID objectId;
}
(SubTag::RootJNIGlobal): {
ID objectId;
ID jniGlobalRefId;
}
(SubTag::RootJNILocal): {
ID objectId;
u32 threadSerialNumber;
u32 frameNumberInStackTrace;
}
(SubTag::RootJavaFrame): {
ID objectId;
u32 threadSerialNumber;
u32 frameNumberInStackTrace;
}
(SubTag::RootNativeStack): {
ID objectId;
u32 threadSerialNumber;
}
(SubTag::RootStickyClass): {
ID objectId;
}
(SubTag::RootThreadBlock): {
ID objectId;
u32 threadSerialNumber;
}
(SubTag::RootMonitorUsed): {
ID objectId;
}
(SubTag::RootThreadObject): {
ID threadObjectId;
u32 threadSerialNumber;
u32 stackTraceSerialNumber;
}
(SubTag::ClassDump): {
ID classObjectId;
u32 stackTraceSerialNumber;
ID superClassObjectId;
ID classLoaderObjectId;
ID signersObjectId;
ID protectionDomainObjectId;
ID reserved[2];
u32 instanceSize;
ConstantPoolArray constantPool;
StaicFieldArray staticFields;
InstanceFieldsArray instanceFields;
}
(SubTag::InstanceDump): {
ID objectId;
u32 stackTraceSerialNumber;
ID classObjectId;
u32 numBytes;
std::mem::Bytes<numBytes> instanceFieldValues;
}
(SubTag::ObjectArrayDump): {
ID arrayObjectId;
u32 stackTraceSerialNumber;
u32 numberOfElements;
ID arrayClassObjectId;
ID elements[numberOfElements];
}
(SubTag::PrimitiveArrayDump): {
ID arrayObjectId;
u32 stackTraceSerialNumber;
u32 numberOfElements;
EntryType type;
BasicType<type> value[numberOfElements];
}
(_): std::error(std::format("Heap Dump Sub Tag {:02X} at 0x{:08X} {}", u32(subTag), $, std::core::array_index()));
}
if ($ - addressof(parent.length) + sizeof(parent.length) >= parent.length)
break;
u8 endTag [[no_unique_address, hidden]];
if (endTag == 0x2C) {
padding[1];
break;
}
};
struct UnloadClass {
u32 classSerialNumber;
};
bitfield AllocSitesFlags {
completeInsteadOfIncremental : 1;
sortedByLineInsteadOfAllocation : 1;
forceGc : 1;
padding : 13;
};
struct AllocSite {
bool arrayIndicator;
u32 classSerialNumber;
u32 stackTraceSerialNumber;
u32 numberOfLiveBytes;
u32 numberOfLiveInstances;
u32 numberOfBytesAllocated;
u32 numberOfInstancesAllocated;
};
struct AllocSites {
AllocSitesFlags flags;
float cutoffRatio;
u32 totalLiveBytes;
u32 totalLiveInstances;
u64 totalBytesAllocated;
u64 totalInstancesAllocated;
u32 numSites;
AllocSite sites[numSites];
};
struct HeapSummary {
u32 totalLiveBytes;
u32 totalLiveInstances;
u64 totalBytesAllocated;
u64 totalInstancesAllocated;
};
struct StartThread {
u32 threadSerialNumber;
ID threadObjectId;
u32 stackTraceSerialNumber;
ID threadNameStringId;
ID threadGroupNameId;
ID threadParentGroupNameId;
};
struct EndThread {
u32 threadSerialNumber;
};
struct Sample {
u32 numberOfSamples;
u32 stackTraceSerialNumber;
};
struct CpuSamples {
u32 numSamples;
Sample samples[numSamples];
};
bitfield ControlSettingsFlags {
allocTraces : 1;
cpuSampling : 1;
padding : 30;
};
struct ControlSettings {
ControlSettingsFlags flags;
u16 stackTraceDepth;
};
struct Record {
Tag tag;
u32 time;
u32 length;
match (tag) {
(Tag::STRING_IN_UTF8):
StringInUTF8 body;
(Tag::LOAD_CLASS):
LoadClass body;
(Tag::UNLOAD_CLASS):
UnloadClass body;
(Tag::STACK_FRAME):
StackFrame body;
(Tag::STACK_TRACE):
StackTrace body;
(Tag::ALLOC_SITES):
AllocSites body;
(Tag::HEAP_SUMMARY):
HeapSummary body;
(Tag::START_THREAD):
StartThread body;
(Tag::END_THREAD):
EndThread body;
(Tag::HEAP_DUMP | Tag::HEAP_DUMP_SEGMENT):
HeapDump heapDumps[while(true)];
(Tag::HEAP_DUMP_END):
std::error("HEAP_DUMP_END Tag without previous HEAP_DUMP or HEAP_DUMP_SEGMENT Tag");
(Tag::CPU_SAMPLES):
CpuSamples body;
(Tag::CONTROL_SETTINGS):
ControlSettings body;
(_):
std::error(std::format("Unknown record type {:02X} at address 0x{:08X}, index {}", u8(tag), addressof(tag), std::core::array_index()));
}
};
struct Header {
char format_name[];
u32 identifier_size;
id_size = identifier_size;
u32 timestamp_high;
u32 time_stamp_low;
};
struct HPROF {
Header header;
Record records[while(!std::mem::eof())];
};
HPROF hprof @ 0x00;

View File

@@ -4,6 +4,10 @@
#pragma endian little
import std.sys;
import std.mem;
import std.string;
import * from bmp as BMP;
import * from png as PNG;
#pragma MIME image/vnd.microsoft.icon
#pragma MIME image/x-icon
@@ -13,38 +17,61 @@ import std.sys;
#pragma MIME application/ico
enum ImageType : u16 {
Icon = 1,
Cursor = 2
Icon = 1,
Cursor = 2
};
struct ICONDIR {
u16 reserved [[hidden]];
ImageType type;
u16 num_images;
u16 reserved [[hidden]];
ImageType type;
u16 num_images;
};
struct BMPData {
u8 data[sizeof(parent.raw_data)] [[hidden, no_unique_address]];
std::mem::Section section = std::mem::create_section("BMP Image " + std::string::to_string(addressof(this)));
std::mem::set_section_size(section, sizeof(data) + 14);
u8 headerData[14] @ 0x00 in section [[hidden]];
headerData[0x00] = 0x42;
headerData[0x01] = 0x4D;
u32 size @ 0x02 in section [[hidden]];
size = std::mem::get_section_size(section);
std::mem::copy_value_to_section(data, section, 0x0E);
BMP image @ 0x00 in section;
u32 offset @ 0x0D in section [[hidden]];
offset = addressof(image.lineData);
};
struct ImageData {
u8 data[parent.image_data_size] [[inline]];
std::mem::Bytes<parent.image_data_size> raw_data [[no_unique_address]];
be u64 png_magic [[hidden, no_unique_address]];
if (png_magic == 0x89504e470d0a1a0a) {
PNG png_image @ $ [[inline, highlight_hidden]];
} else {
BMPData data [[hidden]];
BMP bmp_image @ addressof(this) - 14 [[inline, highlight_hidden]];
}
};
struct ICONDIRENTRY {
u8 width, height;
u8 num_colors;
u8 reserved [[hidden]];
u8 width, height;
u8 num_colors;
u8 reserved [[hidden]];
if (header.type == ImageType::Icon) {
u16 color_planes;
u16 bits_per_pixel;
} else if (header.type == ImageType::Cursor) {
u16 horizontal_hotspot_coordinate;
u16 vertical_hotspot_coordinate;
}
if (header.type == ImageType::Icon) {
u16 color_planes;
u16 bits_per_pixel;
} else if (header.type == ImageType::Cursor) {
u16 horizontal_hotspot_coordinate;
u16 vertical_hotspot_coordinate;
}
u32 image_data_size;
ImageData *image_data : u32;
u32 image_data_size;
u32 image_data_offset;
ImageData image_data @ image_data_offset;
};
ICONDIR header @ 0x00;
ICONDIRENTRY images[header.num_images] @ $;
std::assert(header.reserved == 0x00, "Invalid ICO header");
std::assert(header.reserved == 0x00, "Invalid ICO header");

52
patterns/imah.hexpat Normal file
View File

@@ -0,0 +1,52 @@
#pragma author Hrant (0xZ3R0)
#pragma description DJI Encrypted/Signed Firmware (IM*H)
#pragma endian little
// refs:
// - "Challenges in Dynamic Analysis of Drone Firmware and Its Solutions" (DOI: 10.1109/ACCESS.2024.3425604)
// - "Drone Security and the Mysterious Case of DJIs DroneID" (DOI: 10.14722/ndss.2023.24217)
// - https://github.com/o-gs/dji-firmware-tools
struct imah_chunk_header {
s8 id[4];
u32 offset;
u32 size;
u32 attrib;
u64 address;
u8 reserved[8];
};
struct imah_header {
s8 magic[4];
u32 header_version;
u32 size;
u8 reserved[4];
u32 header_size;
u32 signature_size;
u32 payload_size;
u32 target_size;
u8 os;
u8 arch;
u8 compression;
u8 anti_version;
u32 auth_alg;
u8 auth_key[4];
u8 enc_key[4];
u8 scram_key[16];
s8 name[32];
u8 type[4];
u32 version;
u32 date;
u32 encr_cksum;
u8 reserved2[16];
s8 userdata[16];
u8 entry[8];
u32 plain_cksum;
u32 chunk_num;
u8 payload_digest[32];
};
imah_header header @ 0x00;
imah_chunk_header chunks[header.chunk_num] @ addressof (header) + sizeof (header);
u8 signature[header.signature_size] @ addressof (chunks) + sizeof (chunks);
u8 payload_start @ addressof (signature) + sizeof (signature);

View File

@@ -117,7 +117,10 @@ namespace fmt {
};
fn const_ref(u2 index) {
cp_info info = file.constant_pool[index-1];
return fmt::format_info_const_ref(index, file.constant_pool[index-1]);
};
fn format_info_const_ref(u2 index, ref auto info) {
match(info.tag) {
(1): return info.bytes;
(3): return std::format("{:d} [{:d}]", index, info.bytes);
@@ -140,7 +143,10 @@ namespace fmt {
};
fn const_ref_top(u2 index) {
cp_info info = file.constant_pool[index-1];
return fmt::format_info_const_ref_top(index, file.constant_pool[index-1]);
};
fn format_info_const_ref_top(u2 index, ref auto info) {
match(info.tag) {
(1): return std::format("{:d} [{:s}]", index, info.bytes);
(_): return fmt::const_ref(index);

View File

@@ -145,6 +145,7 @@ struct Segment {
u16 length;
if (marker == Marker::APP0) {
APP0 data;
u8 xtra_data[length - sizeof(length) - sizeof(data)];
} else if (marker == Marker::APP14) {
APP14 data;
} else if (marker == Marker::COM) {

View File

@@ -44,8 +44,15 @@ struct LuaBinaryHeader {
struct LuaString {
u8 size;
if (size > 0) {
char data[size-1];
if (size == 0xff) {
u64 sizeReal;
char data[sizeReal-1];
} else {
char data[size-1];
}
}
}[[format("impl::format_LuaString")]];
struct LuaConstant {

View File

@@ -4,6 +4,7 @@
#pragma MIME application/x-mach-binary
import type.size;
import std.string;
enum Magic : u32 {
_32BitMagic = 0xFEEDFACE,
@@ -367,18 +368,44 @@ struct CommandSegment64 {
u8 data[fileSize] @ fileOffset [[sealed]];
};
struct lc_str { // NB: struct as opposed to it's union def in loader.h
u32 offset;
// It's important to note that the `ptr` field is not used in the Mach-O file itself.
// It is only relevant at runtime when the Mach-O file is loaded into memory.
// The Mach-O file uses the `offset` field to locate the string data within the file.
};
struct Dylib {
lc_str name; // always 24
u32 timestamp;
u32 current_version;
u32 compatibility_version;
};
struct LoadDyLib {
Dylib dylib [[inline]];
};
struct LoadCommand {
Command command;
type::Size<u32> commandSize;
if (command == Command::UUID)
CommandUUID data;
else if (command == Command::Segment)
CommandSegment data;
else if (command == Command::Segment64)
CommandSegment64 data;
else
u8 data[commandSize - 8] [[sealed]];
match (command) {
(Command::UUID): CommandUUID data;
(Command::Segment): CommandSegment data;
(Command::Segment64): CommandSegment64 data;
(Command::LoadDyLib | Command::IdDyLib | Command::LoadWeakDyLib): {
LoadDyLib dylib;
// always 24 bytes in the struct
std::string::NullString dylib_name;
// the 1 is the size of the string terminator
// not using align here since according to the spec this is computed
padding[commandSize - dylib.dylib.name.offset - 1 - std::string::length(dylib_name)];
}
(_): u8 data[commandSize - 8] [[sealed]];
}
};
struct MachO {

37
patterns/mo.hexpat Normal file
View File

@@ -0,0 +1,37 @@
#pragma author dvob
#pragma description GNU Machine Object (MO) files containing translations for gettext
// https://www.gnu.org/software/gettext/manual/html_node/MO-Files.html
import std.core;
import std.io;
struct strPtr {
u32 length;
u32 offset;
char string[length] @ offset;
};
struct file {
be u32 magic;
if ( magic == 0x950412de )
std::core::set_endian(std::mem::Endian::Big);
else if ( magic == 0xde120495)
std::core::set_endian(std::mem::Endian::Little);
else
std::error("Invalid MO Magic!");
u16 majorVersion;
u16 minorVersion;
u32 count;
u32 msgIdOffset;
u32 msgStrOffset;
u32 hashSize;
u32 hashOffset;
u32 hashTable[hashSize] @ hashOffset;
strPtr msgIDTable[count] @ msgIdOffset;
strPtr msgStrTable[count] @ msgStrOffset;
};
file moFile @ 0x0;

View File

@@ -424,7 +424,7 @@ struct MovieBox : BaseBox {
};
struct MediaDataBox : BaseBox {
u8 data[while($ < endOffset)] [[sealed]];
std::mem::Bytes<boxSize - sizeof(size) - sizeof(type)> data;
};
struct Box {

73
patterns/nds.hexpat Normal file
View File

@@ -0,0 +1,73 @@
#pragma author GekySan
#pragma description DS Cartridge Header
#pragma MIME application/x-nintendo-ds-rom
// History
// - 0.1 2025-03-17 GekySan: Initial release. Includes only the header structure.
//
// More information available at:
// - https://problemkaputt.de/gbatek-ds-cartridge-header.htm
enum NDSRegion : u8 {
Normal = 0x00,
China = 0x80,
Korea = 0x40
};
struct NDSHeader {
u8 gameTitle[12] [[comment("Game Title (Uppercase ASCII, padded with 00h)")]];
u8 gameCode[4] [[comment("Gamecode (Uppercase ASCII, NTR-<code>) (0=homebrew)")]];
u8 makerCode[2] [[comment("Makercode (Uppercase ASCII, eg. '01'=Nintendo) (0=homebrew)")]];
u8 unitCode [[comment("Unitcode (00h=NDS, 02h=NDS+DSi, 03h=DSi) (bit1=DSi)")]];
u8 encryptionSeedSelect [[comment("Encryption Seed Select (00..07h, usually 00h)")]];
u8 deviceCapacity [[comment("Device capacity (Chipsize = 128KB SHL nn) (eg. 7 = 16MB)")]];
u8 reserved1[7] [[comment("Reserved (zero filled)")]];
u8 reserved2 [[comment("Reserved (zero) (except used on DSi)")]];
NDSRegion region [[comment("NDS Region (00h=Normal, 80h=China, 40h=Korea) (other on DSi)")]];
u8 romVersion [[comment("ROM Version (usually 00h)")]];
u8 autoStart [[comment("Autostart (Bit2: Skip 'Press Button' after Health and Safety)")]];
u8 arm9RomOffset[4] [[comment("ARM9 rom_offset (4000h and up, align 1000h)")]];
u8 arm9EntryAddress[4] [[comment("ARM9 entry_address (2000000h..23BFE00h)")]];
u8 arm9RamAddress[4] [[comment("ARM9 ram_address (2000000h..23BFE00h)")]];
u8 arm9Size[4] [[comment("ARM9 size (max 3BFE00h)")]];
u8 arm7RomOffset[4] [[comment("ARM7 rom_offset (8000h and up)")]];
u8 arm7EntryAddress[4] [[comment("ARM7 entry_address (2000000h..23BFE00h, or 37F8000h..3807E00h)")]];
u8 arm7RamAddress[4] [[comment("ARM7 ram_address (2000000h..23BFE00h, or 37F8000h..3807E00h)")]];
u8 arm7Size[4] [[comment("ARM7 size (max 3BFE00h, or FE00h)")]];
u8 fntOffset[4] [[comment("File Name Table (FNT) offset")]];
u8 fntSize[4] [[comment("File Name Table (FNT) size")]];
u8 fatOffset[4] [[comment("File Allocation Table (FAT) offset")]];
u8 fatSize[4] [[comment("File Allocation Table (FAT) size")]];
u8 arm9OverlayOffset[4] [[comment("File ARM9 overlay_offset")]];
u8 arm9OverlaySize[4] [[comment("File ARM9 overlay_size")]];
u8 arm7OverlayOffset[4] [[comment("File ARM7 overlay_offset")]];
u8 arm7OverlaySize[4] [[comment("File ARM7 overlay_size")]];
u8 portSetting1[4] [[comment("Port 40001A4h setting for normal commands")]];
u8 portSetting2[4] [[comment("Port 40001A4h setting for KEY1 commands")]];
u8 iconTitleOffset[4] [[comment("Icon/Title offset (0=None) (8000h and up)")]];
u8 secureAreaChecksum[2] [[comment("Secure Area Checksum, CRC-16 of [020h]..00007FFFh")]];
u8 secureAreaDelay[2] [[comment("Secure Area Delay (in 131kHz units)")]];
u8 arm9AutoLoadListRamAddress[4] [[comment("ARM9 Auto Load List Hook RAM Address")]];
u8 arm7AutoLoadListRamAddress[4] [[comment("ARM7 Auto Load List Hook RAM Address")]];
u8 secureAreaDisable[8] [[comment("Secure Area Disable (by encrypted 'NmMdOnly')")]];
u8 totalUsedRomSize[4] [[comment("Total Used ROM size (remaining/unused bytes usually FFh-padded)")]];
u8 romHeaderSize[4] [[comment("ROM Header Size (4000h)")]];
u8 unknown[4] [[comment("Unknown, some rom_offset, or zero? (DSi: slightly different)")]];
u8 reserved3[8] [[comment("Reserved (zero filled; except, [88h..93h] used on DSi)")]];
u8 nandEndOfRomArea[2] [[comment("NAND end of ROM area (in 20000h-byte units)")]];
u8 nandStartOfRWArea[2] [[comment("NAND start of RW area (usually both same address)")]];
u8 reserved4[0x18] [[comment("Reserved (zero filled)")]];
u8 reserved5[0x10] [[comment("Reserved (zero filled; or 'DoNotZeroFillMem'=unlaunch fastboot)")]];
u8 nintendoLogo[0x9C] [[comment("Nintendo Logo (compressed bitmap, same as in GBA Headers)")]];
u8 nintendoLogoChecksum[2] [[comment("Nintendo Logo Checksum, CRC-16 of [0C0h-15Bh], fixed CF56h")]];
u8 headerChecksum[2] [[comment("Header Checksum, CRC-16 of [000h-15Dh]")]];
u8 debugRomOffset[4] [[comment("Debug rom_offset (0=none) (8000h and up)")]];
u8 debugSize[4] [[comment("Debug size (0=none) (max 3BFE00h)")]];
u8 debugRamAddress[4] [[comment("Debug ram_address (0=none) (2400000h..27BFE00h)")]];
u8 reserved6[4] [[comment("Reserved (zero filled) (transferred, and stored, but not used)")]];
u8 reserved7[0x90] [[comment("Reserved (zero filled) (transferred, but not stored in RAM)")]];
u8 reserved8[0xE00] [[comment("Reserved (zero filled) (usually not transferred)")]];
};
NDSHeader NDSHeader @ 0x0000;

View File

@@ -4,6 +4,7 @@
import std.io;
import std.sys;
import type.magic;
struct MOD0 {
char magic[4];
@@ -27,7 +28,7 @@ struct SegmentHeader {
};
struct Header {
char magic[4];
type::Magic<"NRO0"> magic;
u32 version;
u32 file_size;
u32 flags;
@@ -67,7 +68,6 @@ fn is_homebrew() {
Start start @ 0x00;
Header header @ $;
std::assert(header.magic == "NRO0", "Invalid NRO File!");
AssetHeader assets @ header.file_size;

View File

@@ -5,6 +5,7 @@
import std.core;
import std.mem;
import type.magic;
bitfield HeaderType {
Continuation : 1;
@@ -17,7 +18,7 @@ struct SegmentData {
};
struct Ogg {
char capturePattern[4];
type::Magic<"OggS"> capturePattern;
u8 version;
HeaderType headerType;
u64 granulePosition;
@@ -29,4 +30,4 @@ struct Ogg {
SegmentData data[pageSegments];
};
Ogg ogg[while(!std::mem::eof())] @ 0x00;
Ogg ogg[while(!std::mem::eof())] @ 0x00;

223
patterns/optee_ta.hexpat Normal file
View File

@@ -0,0 +1,223 @@
#pragma author WerWolv
#pragma description ARM OPTEE Trusted Application
#pragma endian little
import type.magic;
import type.size;
import type.guid;
import std.mem;
import * from elf as ELF;
enum shdr_img_type : u32 {
SHDR_TA = 0,
SHDR_BOOTSTRAP_TA = 1,
SHDR_ENCRYPTED_TA = 2,
SHDR_SUBKEY = 3
};
enum tee_algorithm : u32 {
TEE_ALG_AES_ECB_NOPAD = 0x10000010,
TEE_ALG_AES_CBC_NOPAD = 0x10000110,
TEE_ALG_AES_CTR = 0x10000210,
TEE_ALG_AES_CTS = 0x10000310,
TEE_ALG_AES_XTS = 0x10000410,
TEE_ALG_AES_CBC_MAC_NOPAD = 0x30000110,
TEE_ALG_AES_CBC_MAC_PKCS5 = 0x30000510,
TEE_ALG_AES_CMAC = 0x30000610,
TEE_ALG_AES_CCM = 0x40000710,
TEE_ALG_AES_GCM = 0x40000810,
TEE_ALG_DES_ECB_NOPAD = 0x10000011,
TEE_ALG_DES_CBC_NOPAD = 0x10000111,
TEE_ALG_DES_CBC_MAC_NOPAD = 0x30000111,
TEE_ALG_DES_CBC_MAC_PKCS5 = 0x30000511,
TEE_ALG_DES3_ECB_NOPAD = 0x10000013,
TEE_ALG_DES3_CBC_NOPAD = 0x10000113,
TEE_ALG_DES3_CBC_MAC_NOPAD = 0x30000113,
TEE_ALG_DES3_CBC_MAC_PKCS5 = 0x30000513,
TEE_ALG_SM4_ECB_NOPAD = 0x10000014,
TEE_ALG_SM4_CBC_NOPAD = 0x10000114,
TEE_ALG_SM4_CTR = 0x10000214,
TEE_ALG_RSASSA_PKCS1_V1_5_MD5 = 0x70001830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA1 = 0x70002830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA224 = 0x70003830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA256 = 0x70004830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA384 = 0x70005830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA512 = 0x70006830,
TEE_ALG_RSASSA_PKCS1_V1_5_MD5SHA1 = 0x7000F830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA3_224 = 0x70008830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA3_256 = 0x70009830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA3_384 = 0x7000A830,
TEE_ALG_RSASSA_PKCS1_V1_5_SHA3_512 = 0x7000B830,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA1 = 0x70212930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA224 = 0x70313930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA256 = 0x70414930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA384 = 0x70515930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA512 = 0x70616930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA3_224 = 0x70818930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA3_256 = 0x70919930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA3_384 = 0x70A1A930,
TEE_ALG_RSASSA_PKCS1_PSS_MGF1_SHA3_512 = 0x70B1B930,
TEE_ALG_RSAES_PKCS1_V1_5 = 0x60000130,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA1 = 0x60210230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA224 = 0x60310230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA256 = 0x60410230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA384 = 0x60510230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA512 = 0x60610230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA3_224 = 0x60810230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA3_256 = 0x60910230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA3_384 = 0x60A10230,
TEE_ALG_RSAES_PKCS1_OAEP_MGF1_SHA3_512 = 0x60B10230,
TEE_ALG_RSA_NOPAD = 0x60000030,
TEE_ALG_DSA_SHA1 = 0x70002131,
TEE_ALG_DSA_SHA224 = 0x70003131,
TEE_ALG_DSA_SHA256 = 0x70004131,
TEE_ALG_DSA_SHA3_224 = 0x70008131,
TEE_ALG_DSA_SHA3_256 = 0x70009131,
TEE_ALG_DSA_SHA3_384 = 0x7000A131,
TEE_ALG_DSA_SHA3_512 = 0x7000B131,
TEE_ALG_SM2_DSA_SM3 = 0x70006045,
TEE_ALG_DH_DERIVE_SHARED_SECRET = 0x80000032,
TEE_ALG_SM2_KEP = 0x60000045,
TEE_ALG_MD5 = 0x50000001,
TEE_ALG_SHA1 = 0x50000002,
TEE_ALG_SHA224 = 0x50000003,
TEE_ALG_SHA256 = 0x50000004,
TEE_ALG_SHA384 = 0x50000005,
TEE_ALG_SHA512 = 0x50000006,
TEE_ALG_SHA3_224 = 0x50000008,
TEE_ALG_SHA3_256 = 0x50000009,
TEE_ALG_SHA3_384 = 0x5000000A,
TEE_ALG_SHA3_512 = 0x5000000B,
TEE_ALG_MD5SHA1 = 0x5000000F,
TEE_ALG_HMAC_MD5 = 0x30000001,
TEE_ALG_HMAC_SHA1 = 0x30000002,
TEE_ALG_HMAC_SHA224 = 0x30000003,
TEE_ALG_HMAC_SHA256 = 0x30000004,
TEE_ALG_HMAC_SHA384 = 0x30000005,
TEE_ALG_HMAC_SHA512 = 0x30000006,
TEE_ALG_HMAC_SM3 = 0x30000007,
TEE_ALG_HMAC_SHA3_224 = 0x30000008,
TEE_ALG_HMAC_SHA3_256 = 0x30000009,
TEE_ALG_HMAC_SHA3_384 = 0x3000000A,
TEE_ALG_HMAC_SHA3_512 = 0x3000000B,
/*
* These are used in the OP-TEE ABI, due to an inconsistency in the v1.1
* specification the wrong values we assumed and now we're stuck with those.
*
* In GP Internal Core API v1.1
* "Table 6-12: Structure of Algorithm Identifier"
* indicates ECDSA have the algorithm "0x41" and ECDH "0x42"
* whereas
* "Table 6-11: List of Algorithm Identifiers" defines
* TEE_ALG_ECDSA_P192 as 0x70001042
*
* We chose to define __OPTEE_TEE_ALG_ECDSA_P192 as 0x70001041 and so on
* to conform to table 6-12.
*/
__OPTEE_ALG_ECDSA_P192 = 0x70001041,
__OPTEE_ALG_ECDSA_P224 = 0x70002041,
__OPTEE_ALG_ECDSA_P256 = 0x70003041,
__OPTEE_ALG_ECDSA_P384 = 0x70004041,
__OPTEE_ALG_ECDSA_P521 = 0x70005041,
__OPTEE_ALG_ECDH_P192 = 0x80001042,
__OPTEE_ALG_ECDH_P224 = 0x80002042,
__OPTEE_ALG_ECDH_P256 = 0x80003042,
__OPTEE_ALG_ECDH_P384 = 0x80004042,
__OPTEE_ALG_ECDH_P521 = 0x80005042,
/* TEE_ALG_ECDSA_P* and TEE_ALG_ECDH_P* are deprecated */
TEE_ALG_ECDSA_P192 = tee_algorithm::TEE_ALG_ECDSA_SHA1,
TEE_ALG_ECDSA_P224 = tee_algorithm::TEE_ALG_ECDSA_SHA224,
TEE_ALG_ECDSA_P256 = tee_algorithm::TEE_ALG_ECDSA_SHA256,
TEE_ALG_ECDSA_P384 = tee_algorithm::TEE_ALG_ECDSA_SHA384,
TEE_ALG_ECDSA_P521 = tee_algorithm::TEE_ALG_ECDSA_SHA512,
TEE_ALG_ECDH_P192 = tee_algorithm::TEE_ALG_ECDH_DERIVE_SHARED_SECRET,
TEE_ALG_ECDH_P224 = tee_algorithm::TEE_ALG_ECDH_DERIVE_SHARED_SECRET,
TEE_ALG_ECDH_P256 = tee_algorithm::TEE_ALG_ECDH_DERIVE_SHARED_SECRET,
TEE_ALG_ECDH_P384 = tee_algorithm::TEE_ALG_ECDH_DERIVE_SHARED_SECRET,
TEE_ALG_ECDH_P521 = tee_algorithm::TEE_ALG_ECDH_DERIVE_SHARED_SECRET,
TEE_ALG_ECDH_DERIVE_SHARED_SECRET = 0x80000042,
TEE_ALG_ECDSA_SHA1 = 0x70001042,
TEE_ALG_ECDSA_SHA224 = 0x70002042,
TEE_ALG_ECDSA_SHA256 = 0x70003042,
TEE_ALG_ECDSA_SHA384 = 0x70004042,
TEE_ALG_ECDSA_SHA512 = 0x70005042,
TEE_ALG_ECDSA_SHA3_224 = 0x70006042,
TEE_ALG_ECDSA_SHA3_256 = 0x70007042,
TEE_ALG_ECDSA_SHA3_384 = 0x70008042,
TEE_ALG_ECDSA_SHA3_512 = 0x70009042,
TEE_ALG_ED25519 = 0x70006043,
TEE_ALG_ED448 = 0x70006044,
TEE_ALG_SM2_PKE = 0x80000046,
TEE_ALG_HKDF = 0x80000047,
TEE_ALG_SM3 = 0x50000007,
TEE_ALG_X25519 = 0x80000044,
TEE_ALG_X448 = 0x80000045,
TEE_ALG_SM4_ECB_PKCS5 = 0x10000015,
TEE_ALG_SM4_CBC_PKCS5 = 0x10000115,
TEE_ALG_ILLEGAL_VALUE = 0xEFFFFFFF,
TEE_ALG_SHA3_224 = 0x50000008,
TEE_ALG_SHA3_256 = 0x50000009,
TEE_ALG_SHA3_384 = 0x5000000A,
TEE_ALG_SHA3_512 = 0x5000000B,
TEE_ALG_SHAKE128 = 0x50000101,
TEE_ALG_SHAKE256 = 0x50000102
};
struct shdr {
type::Magic<"HSTO"> magic;
shdr_img_type img_type;
type::Size<u32> img_size;
tee_algorithm algo;
type::Size<u16> hash_size;
type::Size<u16> sig_size;
std::mem::Bytes<hash_size> hash;
std::mem::Bytes<sig_size> sig;
};
struct shdr_bootstrap_ta {
be type::GUID uuid;
u32 ta_version;
};
enum shdr_enc_key_type : u32 {
SHDR_ENC_KEY_DEV_SPECIFIC = 0,
SHDR_ENC_KEY_CLASS_WIDE = 1,
};
struct shdr_encrypted_ta {
tee_algorithm enc_algo;
shdr_enc_key_type flags;
type::Size<u16> iv_size;
type::Size<u16> tag_size;
std::mem::Bytes<iv_size> iv;
std::mem::Bytes<tag_size> tag;
};
struct OPTEE_TrustedApplication {
shdr shdr;
match (shdr.img_type) {
(shdr_img_type::SHDR_BOOTSTRAP_TA): {
shdr_bootstrap_ta bootstrap_ta_subheader;
}
(shdr_img_type::SHDR_ENCRYPTED_TA): {
shdr_bootstrap_ta bootstrap_ta_subheader;
shdr_encrypted_ta encrypted_ta_subheader;
}
}
if (shdr.img_type == shdr_img_type::SHDR_ENCRYPTED_TA) {
std::mem::Bytes<std::mem::size() - $> encrypted_data;
} else {
ELF elf;
}
};
OPTEE_TrustedApplication trusted_application @ 0x00;

134
patterns/orp.hexpat Normal file
View File

@@ -0,0 +1,134 @@
#pragma description OpenRGB Profile
#pragma magic [ 4F 50 45 4E 52 47 42 5F 50 52 4F 46 49 4C 45 00 ] @ 0x00
import type.magic;
import std.mem;
using Controller;
struct Profile {
type::Magic<"OPENRGB_PROFILE\x00"> signature;
u32 version;
Controller controllers[while(!std::mem::eof())];
};
Profile profile @ 0x00;
using Mode;
using Zone;
using led;
using RGBColor;
using _LedAltName;
struct Controller {
u32 size;
s32 device_type;
u16 name_len;
char name[name_len];
u16 vendor_len;
char vendor[vendor_len];
u16 description_len;
char description[description_len];
if (parent.version >= 1) {
u16 version_len;
char version[version_len];
}
u16 serial_len;
char serial[serial_len];
u16 location_len;
char location[location_len];
u16 num_modes;
u32 active_mode;
Mode modes[num_modes];
u16 num_zones;
Zone zones[num_zones];
u16 num_leds;
led leds[num_leds];
u16 num_colors;
RGBColor colors[num_colors];
if (parent.version >= 5) {
u16 num_led_alt_names;
_LedAltName led_alt_names[num_led_alt_names];
u32 flags;
}
};
struct Mode {
u16 name_len;
char name[name_len];
u32 value;
u32 flags;
u32 speed_min;
u32 speed_max;
if (parent.parent.version >= 3) {
u32 brightness_min;
u32 brightness_max;
}
u32 colors_min;
u32 colors_max;
u32 speed;
if (parent.parent.version >= 3) {
u32 brightness;
}
u32 direction;
u32 color_mode;
u16 colors_len;
RGBColor colors[colors_len];
};
using zone_type = s32;
using matrix_map_type;
using segment;
struct Zone {
u16 name_len;
char name[name_len];
zone_type type;
u32 leds_min;
u32 leds_max;
u32 leds_count;
u16 matrix_len;
matrix_map_type matrix[matrix_len];
if (parent.parent.version >= 4) {
u16 segments_len;
segment segments[segments_len];
}
if (parent.parent.version >= 5) {
u32 flags;
}
};
struct led {
u16 name_len;
char name[name_len];
u32 value;
};
struct RGBColor {
u8 red [[color("FF0000")]];
u8 green [[color("00FF00")]];
u8 blue [[color("0000FF")]];
u8 alpha;
} [[
color(std::format("{:02X}{:02X}{:02X}{:02X}", red, green, blue, alpha)),
hex::inline_visualize("color", red, green, blue, alpha)
]];
struct _LedAltName {
u16 name_len;
char name[name_len];
};
struct matrix_map_type {
u32 height;
u32 width;
u32 map[width * height];
};
struct segment {
u16 name_len;
char name[name_len];
zone_type type;
u32 start_idx;
u32 leds_count;
};

530
patterns/pcapng.hexpat Normal file
View File

@@ -0,0 +1,530 @@
#pragma description PcapNG
#pragma MIME application/vnd.tcpdump.pcapng
#pragma author 5h4rrK
import std.mem;
import std.core;
import std.math;
import std.io;
#define MAX_VALUE_U64 0xffffffffffffffff
fn format_version(auto version){
return std::format(
"{}.{}", version.major, version.minor
);
};
fn is_valid_len(auto val){
if (val == MAX_VALUE_U64){
return std::format("[Not Specified]");
}
else{
return val;
}
};
enum PcapOrder : u32{
Little = 0x1a2b3c4d,
Big = 0x4d3c2b1a,
};
struct Version {
u16 major;
u16 minor;
};
enum BlockType : u32{
// Mandatory Blocks
SectionHeader = 0x0A0D0D0A,
InterfaceDesc = 0x00000001,
// Optional Blocks
SimplePacket = 0x00000003,
NameResolution = 0x00000004,
InterfaceStats = 0x00000005,
EnhancedPacket = 0x00000006,
Custom = 0x00000BAD,
CustomNoCopy = 0x40000BAD,
Systemd = 0x00000009,
Decryption = 0x0000000A,
// Obsolete Block
Packet = 0x00000002,
// Reserved & Misc
Reserved = 0x00000000,
IRIGTimestamp = 0x00000007,
AFDXInfo = 0x00000008,
// Hone Project Specific
HoneMachine = 0x00000101,
HoneConnEvent = 0x00000102,
// Sysdig Specific
SysdigMachine = 0x00000201,
SysdigProcessV1 = 0x00000202,
SysdigFDList = 0x00000203,
SysdigEvent = 0x00000204,
SysdigIFList = 0x00000205,
SysdigUserList = 0x00000206,
SysdigProcessV2 = 0x00000207,
SysdigEventFL = 0x00000208,
SysdigProcessV3 = 0x00000209,
SysdigProcessV4 = 0x00000210,
SysdigProcessV5 = 0x00000211,
SysdigProcessV6 = 0x00000212,
SysdigProcessV7 = 0x00000213,
};
enum LinkType : u16 {
LINKTYPE_NULL = 0,
LINKTYPE_ETHERNET = 1,
LINKTYPE_AX25 = 3,
LINKTYPE_IEEE802_5 = 6,
LINKTYPE_ARCNET_BSD = 7,
LINKTYPE_SLIP = 8,
LINKTYPE_PPP = 9,
LINKTYPE_FDDI = 10,
LINKTYPE_PPP_HDLC = 50,
LINKTYPE_PPP_ETHER = 51,
LINKTYPE_ATM_RFC1483 = 100,
LINKTYPE_RAW = 101,
LINKTYPE_C_HDLC = 104,
LINKTYPE_IEEE802_11 = 105,
LINKTYPE_FRELAY = 107,
LINKTYPE_LOOP = 108,
LINKTYPE_LINUX_SLL = 113,
LINKTYPE_LTALK = 114,
LINKTYPE_PFLOG = 117,
LINKTYPE_IEEE802_11_PRISM = 119,
LINKTYPE_IP_OVER_FC = 122,
LINKTYPE_SUNATM = 123,
LINKTYPE_IEEE802_11_RADIOTAP = 127,
LINKTYPE_ARCNET_LINUX = 129,
LINKTYPE_APPLE_IP_OVER_IEEE1394 = 138,
LINKTYPE_MTP2_WITH_PHDR = 139,
LINKTYPE_MTP2 = 140,
LINKTYPE_MTP3 = 141,
LINKTYPE_SCCP = 142,
LINKTYPE_DOCSIS = 143,
LINKTYPE_LINUX_IRDA = 144,
LINKTYPE_IEEE802_11_AVS = 163,
LINKTYPE_BACNET_MS_TP = 165,
LINKTYPE_PPP_PPPD = 166,
LINKTYPE_GPRS_LLC = 169,
LINKTYPE_GPF_T = 170,
LINKTYPE_GPF_F = 171,
LINKTYPE_LINUX_LAPD = 177,
LINKTYPE_MFR = 182,
LINKTYPE_BLUETOOTH_HCI_H4 = 187,
LINKTYPE_USB_LINUX = 189,
LINKTYPE_PPI = 192,
LINKTYPE_IEEE802_15_4_WITHFCS = 195,
LINKTYPE_SITA = 196,
LINKTYPE_ERF = 197,
LINKTYPE_BLUETOOTH_HCI_H4_WITH_PHDR = 201,
LINKTYPE_AX25_KISS = 202,
LINKTYPE_LAPD = 203,
LINKTYPE_PPP_WITH_DIR = 204,
LINKTYPE_C_HDLC_WITH_DIR = 205,
LINKTYPE_FRELAY_WITH_DIR = 206,
LINKTYPE_LAPB_WITH_DIR = 207,
LINKTYPE_IPMB_LINUX = 209,
LINKTYPE_FLEXRAY = 210,
LINKTYPE_IEEE802_15_4_NONASK_PHY = 215,
LINKTYPE_USB_LINUX_MMAPPED = 220,
LINKTYPE_FC_2 = 224,
LINKTYPE_FC_2_WITH_FRAME_DELIMS = 225,
LINKTYPE_IPNET = 226,
LINKTYPE_CAN_SOCKETCAN = 227,
LINKTYPE_IPV4 = 228,
LINKTYPE_IPV6 = 229,
LINKTYPE_IEEE802_15_4_NOFCS = 230,
LINKTYPE_DBUS = 231,
LINKTYPE_DVB_CI = 235,
LINKTYPE_MUX27010 = 236,
LINKTYPE_STANAG_5066_D_PDU = 237,
LINKTYPE_NFLOG = 239,
LINKTYPE_NETANALYZER = 240,
LINKTYPE_NETANALYZER_TRANSPARENT = 241,
LINKTYPE_IPOIB = 242,
LINKTYPE_MPEG_2_TS = 243,
LINKTYPE_NG40 = 244,
LINKTYPE_NFC_LLCP = 245,
LINKTYPE_INFINIBAND = 247,
LINKTYPE_SCTP = 248,
LINKTYPE_USBPCAP = 249,
LINKTYPE_RTAC_SERIAL = 250,
LINKTYPE_BLUETOOTH_LE_LL = 251,
LINKTYPE_NETLINK = 253,
LINKTYPE_BLUETOOTH_LINUX_MONITOR = 254,
LINKTYPE_BLUETOOTH_BREDR_BB = 255,
LINKTYPE_BLUETOOTH_LE_LL_WITH_PHDR = 256,
LINKTYPE_PROFIBUS_DL = 257,
LINKTYPE_PKTAP = 258,
LINKTYPE_EPON = 259,
LINKTYPE_IPMI_HPM_2 = 260,
LINKTYPE_ZWAVE_R1_R2 = 261,
LINKTYPE_ZWAVE_R3 = 262,
LINKTYPE_WATTSTOPPER_DLM = 263,
LINKTYPE_ISO_14443 = 264,
LINKTYPE_RDS = 265,
LINKTYPE_USB_DARWIN = 266,
LINKTYPE_SDLC = 268,
LINKTYPE_LORATAP = 270,
LINKTYPE_VSOCK = 271,
LINKTYPE_NORDIC_BLE = 272,
LINKTYPE_DOCSIS31_XRA31 = 273,
LINKTYPE_ETHERNET_MPACKET = 274,
LINKTYPE_DISPLAYPORT_AUX = 275,
LINKTYPE_LINUX_SLL2 = 276,
LINKTYPE_OPENVIZSLA = 278,
LINKTYPE_EBHSCR = 279,
LINKTYPE_VPP_DISPATCH = 280,
LINKTYPE_DSA_TAG_BRCM = 281,
LINKTYPE_DSA_TAG_BRCM_PREPEND = 282,
LINKTYPE_IEEE802_15_4_TAP = 283,
LINKTYPE_DSA_TAG_DSA = 284,
LINKTYPE_DSA_TAG_EDSA = 285,
LINKTYPE_ELEE = 286,
LINKTYPE_Z_WAVE_SERIAL = 287,
LINKTYPE_USB_2_0 = 288,
LINKTYPE_ATSC_ALP = 289,
LINKTYPE_ETW = 290
};
enum SHBType : u16{
EndOfOpt = 0x0,
Hardware = 0x2,
OS = 0x3,
Application = 0x4,
EndOfOpt = 0x0
};
enum IDHType : u16 {
EndOfOpt = 0,
Name = 2, // UTF-8 device name (e.g., "eth0")
Description = 3, // UTF-8 device description
IPv4Addr = 4, // IPv4 address + netmask (8 bytes)
IPv6Addr = 5, // IPv6 address + prefix len (17 bytes)
MACAddr = 6, // MAC address (6 bytes)
EUIAddr = 7, // EUI-64 address (8 bytes)
Speed = 8, // Interface speed (bps, 8 bytes)
TimestampRes = 9, // Timestamp resolution (1 byte)
TimeZone = 10, // Time zone (4 bytes)
Filter = 11, // Capture filter string
OS = 12, // OS name (UTF-8 string)
FCSLength = 13, // Frame Check Sequence length (1 byte)
TimestampOffset = 14, // Timestamp offset (8 bytes)
Hardware = 15, // Variable length
TxSpeed = 16, // 8bytes
RxSpeed = 17 // 8bytes
};
enum NameResolutionType: u16 {
EndOfRecord = 0x00,
IPv4 = 0x01,
IPv6 = 0x02
};
enum InterfaceStatsType : u16 {
EndOfOpt = 0,
Comment = 1, // isb_comment
StartTime = 2, // isb_starttime
EndTime = 3, // isb_endtime
IfRecv = 4, // isb_ifrecv
IfDrop = 5, // isb_ifdrop
FilterAccept = 6, // isb_filteraccept
OSdrop = 7, // isb_osdrop
UserDeliver = 8, // isb_usrdeliv
};
enum EnhancedPacketType : u16 {
EndOfOpt = 0x0,
Flags = 0x2,
Hash = 0x3,
DropCount = 0x4,
PacketId = 0x5,
Queue = 0x6,
Verdict = 0x7,
};
enum PacketBlockType : u16 {
Flags = 0x2,
Hash = 0x3,
EndOfOpt = 0x0,
};
struct Option {
if (this.parent.block_type == BlockType::SectionHeader) {
SHBType option_type;
} else if(this.parent.block_type == BlockType::InterfaceDesc) {
IDHType option_type;
} else if (this.parent.block_type == BlockType::NameResolution) {
NameResolutionType record_type[[name("record")]];
} else if (this.parent.block_type == BlockType::InterfaceStats){
InterfaceStatsType option_type;
} else if (this.parent.block_type == BlockType::EnhancedPacket) {
EnhancedPacketType option_type;
} else if (this.parent.block_type == BlockType::Packet){
PacketBlockType option_type;
}
u16 option_len;
if (option_len > 0){
char data[option_len];
} else{
return;
}
// u8 pad_size = (4 - ( $ % 4 ));
u8 pad_size = (4 - ( $ % 4 )) % 4;
// if(this.parent.block_type == BlockType::InterfaceStats) {
// std::print("Current Pos {:#x} {:#x}",$, pad_size);
// }
$ = $ + pad_size;
};
struct SectionHeaderBlock {
BlockType block_type;
u32 length;
PcapOrder sectionbyteorder;
Version version [[name("Version"), format("format_version")]];
u64 section_len [[name("SectionLen"), format("is_valid_len")]];
u64 prev_pos = $;
Option options [ while( $ < (prev_pos + length - 28))];
u32 block_length [[name("BlockLen")]];
};
struct NameResolutionBlock {
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
u64 prev_pos = $;
Option records [ while($ < (prev_pos + block_len1 - 12)) ];
u32 block_len2[[name("BlockLen2")]];
};
struct SystemJournalExportBlock{
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
char data[block_len1];
$ = $ + (4 - ($ % 4 ) ); // Padding
u32 block_len2[[name("BlockLen2")]];
};
struct CustomBlock{
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
u32 pen[[name("PrivateEnterpriseNumber")]];
char data[block_len1];
u64 prev_pos = $;
$ = $ + (4 - ($ % 4 ) ); // Padding
Option options [while($ < (prev_pos + block_len1 - 16))] [[name("Options")]];
u32 block_len2[[name("BlockLen2")]];
};
struct InterfaceBlock{
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
LinkType link_type [[name("LinkType")]];
u16 reserved [[name("Reserved")]];
u32 snap_len [[name("SnapLen")]];
u64 prev_pos = $;
Option options [ while( $ < (prev_pos + block_len1 - 20))];
u32 block_len2[[name("BlockLen2")]];
};
struct EnhancedPacketBlock{
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
u32 interface_id[[name("InterfaceID")]];
u64 timestamp[[name("Timestamp")]];
u32 captured_len[[name("CapturedLen")]];
u32 pkt_len[[name("PacketLen")]];
char data[captured_len][[name("Data")]];
// 32 (BlockType + BlockLen1 + InterfaceID + Timestamp + CapturedLen + PacketLen + BlockLen2)
$ = $ + (block_len1 - captured_len - 32);
u64 prev_pos = $;
// Current Pos - Prev Pos - 4 (BlockLen2)
// std::print("Option Size :: {:#x}", ($ - prev_pos));
if (($ - prev_pos ) > 0) {
Option options [ while( $ < ( prev_pos + ($ - prev_pos ) ) ) ];
}
u32 block_len2[[name("BlockLen2")]];
};
struct SimplePacketBlock {
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
u32 pkt_len[[name("PacketLen")]];
char data[[name("Data")]];
u8 pad_size = (4 - ( $ % 4 )) % 4;
$ = $ + pad_size;
u32 block_len2[[name("BlockLen2")]];
};
// Obsolete!
struct PacketBlock {
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
u16 interface_id[[name("InterfaceID")]];
u16 drop_count[[name("DropCount")]];
u64 timestamp[[name("Timestamp")]];
u32 captured_len[[name("CapturedLen")]];
u32 pkt_len[[name("PacketLen")]];
if (block_len1 > 32){
char data[captured_len];
}
u32 block_len2[[name("BlockLen2")]];
};
enum CompressionType: u8 {
Uncompressed = 0x0,
LempelZiv = 0x1,
Gzip = 0x2
};
enum EncryptionType : u32 {
TLS = 0x544c534b, /* TLS Key Log */
SSH = 0x5353484b, /* SSH Key Log */
WIREGUARD = 0x57474b4c, /* WireGuard Key Log */
ZIGBEE_NWK_KEY = 0x5a4e574b, /* Zigbee NWK Key */
ZIGBEE_APS_KEY = 0x5a415053 /* Zigbee APS Key */
};
enum FixedLengthType : u8 {
// Experimental
};
// Experimental
struct CompressionBlock{
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
CompressionType comp_type [[name("Compresssion")]];
char data[block_len1];
u32 block_len2[[name("BlockLen2")]];
};
struct EncryptionBlock{
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
EncryptionType comp_type [[name("Encryption")]];
u32 secrets_len[[name("SecretsLength")]];
char data[secrets_len];
padding[-$ & 3];
// https://ietf-opsawg-wg.github.io/draft-ietf-opsawg-pcap/draft-ietf-opsawg-pcapng.html#section-4.7-6.6.1
// No DSB-specific options are currently defined
// Option options;
u32 block_len2[[name("BlockLen2")]];
};
// Experimental
struct FixedLengthBlock {
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
FixedLengthType comp_type [[name("FixedLength")]];
char data[block_len1];
u32 block_len2[[name("BlockLen2")]];
};
struct InterfaceStatsBlock {
BlockType block_type[[name("BlockType")]];
u32 block_len1[[name("BlockLen1")]];
u32 interface_id[[name("InterfaceID")]];
u64 timestamp[[name("Timestamp")]];
if(block_len1 > 24) {
u64 prev_pos = $;
Option options [ while( $ < (prev_pos + block_len1 - 24))];
}
u32 block_len2[[name("BlockLen2")]];
};
struct PCAPng{
if (std::mem::eof()){
break;
}
if (order == PcapOrder::Little){
le u32 block_type = std::mem::read_unsigned($, 0x4, std::mem::Endian::Little);
} else {
be u32 block_type = std::mem::read_unsigned($, 0x4, std::mem::Endian::Big);
}
// std::print("{} , {:#x}, {:#x}", "Parsing", $, block_type);
// std::print("Parsing : Offset({:#x}), Order({:#x}), BlockType({:#x})", $, order, block_type);
if (block_type == BlockType::SectionHeader) {
if (order == PcapOrder::Little) {
le SectionHeaderBlock SHB;
} else {
be SectionHeaderBlock SHB;
}
} else if (block_type == BlockType::InterfaceDesc) {
if (order == PcapOrder::Little) {
le InterfaceBlock IDH;
} else {
be InterfaceBlock IDH;
}
} else if (block_type == BlockType::EnhancedPacket) {
if (order == PcapOrder::Little) {
le EnhancedPacketBlock EBP;
} else {
be EnhancedPacketBlock EBP;
}
} else if (block_type == BlockType::NameResolution) {
if (order == PcapOrder::Little) {
le NameResolutionBlock NRB;
} else {
be NameResolutionBlock NRB;
}
} else if (block_type == BlockType::InterfaceStats) {
if (order == PcapOrder::Little) {
le InterfaceStatsBlock ISB;
} else {
be InterfaceStatsBlock ISB;
}
} else if (block_type == BlockType::SimplePacket) {
if (order == PcapOrder::Little) {
le SimplePacketBlock SPB;
} else {
be SimplePacketBlock SPB;
}
} else if (block_type == BlockType::Packet) {
if (order == PcapOrder::Little) {
le PacketBlock PB;
} else {
be PacketBlock PB;
}
} else if (block_type == BlockType::Custom) {
if (order == PcapOrder::Little) {
le CustomBlock CB;
} else {
be CustomBlock CB;
}
} else if (block_type == BlockType::CustomNoCopy) {
if (order == PcapOrder::Little) {
le CustomBlock CBN;
} else {
be CustomBlock CBN;
}
} else if (block_type == BlockType::Decryption) {
if (order == PcapOrder::Little) {
le EncryptionBlock DSB;
} else {
be EncryptionBlock DSB;
}
} else {
std::print("Unknown BlockType at offset {:#x}\n", $);
break;
}
} [[inline]];
u32 order = std::mem::read_unsigned(0x8, 0x4);
PCAPng PcapNG [while(!std::mem::eof())] @0x00;

View File

@@ -396,7 +396,9 @@ struct ExportsTable {
if (directoryTable.ordinalTableRVA > relativeVirtualDifference()) {
u16 exportOrdinalTable[directoryTable.namePointersAmount] @ directoryTable.ordinalTableRVA - relativeVirtualDifference();
}
char imageName[] @ directoryTable.imageNameRVA - relativeVirtualDifference() [[format("formatNullTerminatedString")]];
if (directoryTable.imageNameRVA > relativeVirtualDifference()) {
char imageName[] @ directoryTable.imageNameRVA - relativeVirtualDifference() [[format("formatNullTerminatedString")]];
}
$ = addressof(this)+coffHeader.optionalHeader.directories[0].size;
};

444
patterns/pef.hexpat Normal file
View File

@@ -0,0 +1,444 @@
#pragma author domin568
#pragma description Preffered Executable Format executable (for Mac OS 7.1.2 - Mac OS 10.4 / BeOS)
#pragma endian big
#pragma pattern_limit 1000000
import std.core;
import std.sys;
import std.math;
struct PEFContainerHeader
{
char tag1 [4] [[comment("designates that the container uses an Apple-defined format. This field must be set to Joy! in ASCII.")]];
char tag2 [4] [[comment("identifies the type of container (currently set to peff in ASCII")]];
char architecture [4] [[comment("indicates the architecture type that the container was generated for. This field holds the ASCII value pwpc for the PowerPC CFM implementation or m68k for CFM-68K.")]];
u32 formatVersion [[comment("indicates the version of PEF used in the container. The current version is 1.")]];
u32 dateTimeStamp [[comment("indicates when the PEF container was created. The stamp follows the Macintosh time-measurement scheme (that is, the number of seconds measured from January 1, 1904)")]];
u32 oldDefVersion [[comment("contain version information that the Code Fragment Manager uses to check shared library compatibility")]];
u32 oldImpVersion [[comment("contain version information that the Code Fragment Manager uses to check shared library compatibility")]];
u32 currentVersion [[comment("contain version information that the Code Fragment Manager uses to check shared library compatibility")]];
u16 sectionCount [[comment("indicates the total number of sections contained in the container.")]];
u16 instSectionCount [[comment("indicates the number of instantiated sections. Instantiated sections contain code or data that are required for execution.")]];
u32 reservedA;
};
enum SectionKind :u8
{
kPEFCodeSection = 0, // Contains read-only executable code in an uncompressed binary format. A container can have any number of code sections. Code sections are always shared.
kPEFUnpackedDataSection = 1, // Contains uncompressed, initialized, read/write data followed by zero-initialized read/write data.
kPEFPackedDataSection = 2, // Contains read/write data initialized by a pattern specification contained in the sections contents. The contents essentially contain a small program that tells the Code Fragment Manager how to initialize the raw data in memory.
kPEFConstantSection = 3, // Contains uncompressed, initialized, read-only data. A container can have any number of constant sections, and they are implicitly shared.
kPEFLoaderSection = 4, // Contains information about imports, exports, and entry points. A container can have only one loader section.
kPEFDebugSection = 5, // Reserved for future use
kPEFExecDataSection = 6, // Contains information that is both executable and modifiable. For example, this section can store code that contains embedded data. A container can have any number of executable data sections, each with a different sharing option.
kPEFExceptionSection = 7, // Reserved for future use
kPEFTracebackSection = 8 // Reserved for future use
};
enum ShareKind : u8
{
Process_Share = 1, //Indicates that the section is shared within a process, but a fresh copy is created for different processes.
Global_Share = 4, // Indicates that the section is shared between all processes in the system.
Protected_Share = 5 // Indicates that the section is shared between all processes, but is protected. Protected sections are read/write in privileged mode and read-only in user mode. This option is not available in System 7.
};
struct PEFSectionHeader
{
s32 nameOffset [[comment("holds the offset from the start of the section name table to the location of the section name. The name of the section is stored as a C-style null-terminated character string.")]];
u32 defaultAddress [[comment("indicates the preferred address (as designated by the linker) at which to place the sections instance. If the Code Fragment Manager can place the instance in the preferred memory location, the load-time and link-time addresses are identical and no internal relocations need to be performed.")]];
u32 totalSize [[comment("indicates the size, in bytes, required by the sections contents at execution time. For a code section, this size is merely the size of the executable code. For a data section, this size indicates the sum of the size of the initialized data plus the size of any zero-initialized data. Zero-initialized data appears at the end of a sections contents and its length is exactly the difference of the totalSize and unpackedSize values. For noninstantiated sections, this field is ignored.")]];
u32 unpackedSize [[comment("is the size of the sections contents that is explicitly initialized from the container. For code sections, this field is the size of the executable code. For an unpacked data section, this field indicates only the size of the initialized data. For packed data this is the size to which the compressed contents expand. The unpackedSize value also defines the boundary between the explicitly initialized portion and the zero-initialized portion.")]];
u32 packedSize [[comment("indicates the size, in bytes, of a sections contents in the container. For code sections, this field is the size of the executable code. For an unpacked data section, this field indicates only the size of the initialized data. For a packed data section this field is the size of the pattern description contained in the section.")]];
u32 containerOffset [[comment("contains the offset from the beginning of the container to the start of the sections contents. Packed data sections and the loader section should be 4-byte aligned. Code sections and data sections that are not packed should be at least 16-byte aligned.")]];
SectionKind sectionKind [[comment("indicates the type of section as well as any special attributes. Note that instantiated read-only sections cannot have zero-initialized extensions.")]];
ShareKind shareKind [[comment("controls how the section information is shared among processes by the Code Fragment Manager.")]];
u8 alignment [[comment("indicates the desired alignment for instantiated sections in memory as a power of 2. A value of 0 indicates 1-byte alignment, 1 indicates 2-byte (halfword) alignment, 2 indicates 4-byte (word) alignment, and so on. Note that this field does not indicate the alignment of raw data relative to a container. The Code Fragment Manager does not support this field under System 7.")]];
u8 reservedA;
u8 data[packedSize] @ containerOffset;
};
struct String {
char value[];
} [[sealed, format("format_string")]];
fn format_string(String string)
{
return string.value;
};
struct PEFLoaderInfoHeader
{
s32 mainSection [[comment("specifies the number of the section in this container that contains the main symbol. If the fragment does not have a main symbol, this field is set to -1.")]];
u32 mainOffset [[comment("indicates the offset (in bytes) from the beginning of the section to the main symbol.")]];
s32 initSection [[comment("contains the number of the section containing the initialization functions transition vector. If no initialization function exists, this field is set to -1.")]];
u32 initOffset [[comment("indicates the offset (in bytes) from the beginning of the section to the initialization functions transition vector.")]];
s32 termSection [[comment("contains the number of the section containing the termination routines transition vector. If no termination routine exists, this field is set to -1.")]];
u32 termOffset [[comment("indicates the offset (in bytes) from the beginning of the section to the termination routines transition vector.")]];
u32 importedLibraryCount [[comment("The importedLibraryCount field (4 bytes) indicates the number of imported libraries.")]];
u32 totalImportedSymbolCount [[comment("indicates the total number of imported symbols.")]];
u32 relocSectionCount [[comment("indicates the number of sections containing load-time relocations.")]];
u32 relocInstrOffset [[comment("indicates the offset (in bytes) from the beginning of the loader section to the start of the relocations area.")]];
u32 loaderStringsOffset [[comment("Indicates the offset (in bytes) from the beginning of the loader section to the start of the loader string table.")]];
u32 exportHashOffset [[comment("indicates the offset (in bytes) from the beginning of the loader section to the start of the export hash table. The hash table should be 4-byte aligned with padding added if necessary.")]];
u32 exportHashTablePower [[comment("indicates the number of hash index values (that is, the number of entries in the hash table). The number of entries is specified as a power of two. For example, a value of 0 indicates one entry, while a value of 2 indicates four entries. If no exports exist, the hash table still contains one entry, and the value of this field is 0.")]];
u32 exportedSymbolCount [[comment("indicates the number of symbols exported from this container.")]];
};
enum importedLibraryOption :u8
{
kDefault = 0x00, /* Bits for the PEFImportedLibrary options field.*/
kPEFWeakImportLibMask = 0x40, /* The imported library is allowed to be missing.*/
kPEFInitLibBeforeMask = 0x80 /* The imported library must be initialized first.*/
};
struct PEFImportedLibrary
{
u32 nameOffset [[comment("Indicates the offset (in bytes) from the beginning of the loader string table to the start of the null-terminated library name.")]];
u32 oldImpVersion [[comment("Provide version information for checking the compatibility of the imported library.")]];
u32 currentVersion [[comment("Provide version information for checking the compatibility of the imported library.")]];
u32 importedSymbolCount [[comment("Indicates the number of symbols imported from this library.")]];
u32 firstImportedSymbol [[comment("Holds the (zero-based) index of the first entry in the imported symbol table for this library.")]];
importedLibraryOption options [[comment("The high-order bit (mask 0x80) controls the order that the import libraries are initialized. If set to 0, the default initialization order is used, which specifies that the Code Fragment Manager should try to initialize the import library before the fragment that imports it. When set to 1, the import library must be initialized before the client fragment. The next bit (mask 0x40) controls whether the import library is weak. When set to 1 (weak import), the Code Fragment Manager continues preparation of the client fragment (and does not generate an error) even if the import library cannot be found. If the import library is not found, all imported symbols from that library have their addresses set to 0. You can use this information to determine whether a weak import library is actually present.")]];
u8 reservedA [[comment("Reserved and must be set to 0.")]];
u16 reservedB [[comment("Reserved and must be set to 0.")]];
} [[format("format_name")]];
enum importSymbolClass :u8
{
kPEFCodeSymbol = 0, // A code address
kPEFDataSymbol = 1, // A data address
kPEFTVectSymbol = 2, // A standard procedure pointer
kPEFTOCSymbol = 3, // A direct data area (Table of Contents ) symbol
kPEFGlueSymbol = 4, // A linker-inserted glue symbol
};
struct PEFImportSymbol
{
importSymbolClass class [[comment("Indicates the class of the imported symbol.")]];
u24 nameOffset [[comment("Indicates the offset (in bytes) from the beginning of the loader string table to the null-terminated name of the symbol")]];
} [[format("format_name")]];
struct PEFLoaderRelocationHeader
{
u16 sectionIndex [[comment("Designates the section number to which this relocation header refers.")]];
u16 reservedA [[comment("Reserved")]];
u32 relocCount [[comment("Indicates the number of 16-bit relocation blocks for this section.")]];
u32 firstRelocOffset [[comment("Indicates the byte offset from the start of the relocations area to the first relocation instruction for this section.")]];
};
enum relocationOpcode : u8
{
kPEFRelocBySectDWithSkip = 0x00, /* Binary: 00x_xxxx*/
kPEFRelocBySectC = 0x20, /* Binary: 010_0000, group is "RelocRun"*/
kPEFRelocBySectD = 0x21, /* Binary: 010_0001*/
kPEFRelocTVector12 = 0x22, /* Binary: 010_0010*/
kPEFRelocTVector8 = 0x23, /* Binary: 010_0011*/
kPEFRelocVTable8 = 0x24, /* Binary: 010_0100*/
kPEFRelocImportRun = 0x25, /* Binary: 010_0101*/
kPEFRelocSmByImport = 0x30, /* Binary: 011_0000, group is "RelocSmIndex"*/
kPEFRelocSmSetSectC = 0x31, /* Binary: 011_0001*/
kPEFRelocSmSetSectD = 0x32, /* Binary: 011_0010*/
kPEFRelocSmBySection = 0x33, /* Binary: 011_0011*/
kPEFRelocIncrPosition = 0x40, /* Binary: 100_0xxx*/
kPEFRelocSmRepeat = 0x48, /* Binary: 100_1xxx*/
kPEFRelocSetPosition = 0x50, /* Binary: 101_000x*/
kPEFRelocLgByImport = 0x52, /* Binary: 101_001x*/
kPEFRelocLgRepeat = 0x58, /* Binary: 101_100x*/
kPEFRelocLgSetOrBySection = 0x5A, /* Binary: 101_101x*/
kPEFRelocUndefinedOpcode = 0xFF /* Used in masking table for all undefined values.*/
};
// "RelocBySectDWithSkip" (DDAT)
bitfield RelocBySectDWithSkip
{
op : 2; // 00
skipCount : 8;
relCount : 6;;
};
// "RelocBySectC" (CODE)
// "RelocBySectD" (DATA)
// "RelocTVector12" (DESC)
// "RelocTVector8" (DSC2)
// "RelocVTable8" (VTBL)
// "RelocImportRun" (SYMR)
bitfield RelocRunGroup
{
op : 3; // 010
subOp : 4;
runLength : 9;
};
// "RelocSmByImport" (SYMB)
// "RelocSmSetSectC" (CDIS)
// "RelocSmSetSectD" (DTIS)
// "RelocSmBySection" (SECN)
bitfield RelocSmIndexGroup
{
op : 3; // 011
subOp : 4;
index : 9;
};
// "RelocIncrPosition" (DELT)
bitfield RelocIncrPosition
{
op : 4; // 1000
offset : 12;
};
// "RelocSmRepeat" (RPT)
bitfield RelocSmRepeat
{
op : 4; // 1001
chnks : 4;
repeatCount : 8;
};
// "RelocSetPosition" (LABS)
bitfield RelocSetPosition
{
op : 6; // 101000
offset : 26;
};
// "RelocLgByImport" (LSYM)
bitfield RelocLgByImport
{
op : 6; // 101001
index : 26;
};
// "RelocLgRepeat" (LRPT)
bitfield RelocLgRepeat
{
op : 6; // 101100
chnks : 4;
repeatCount : 22;
};
// "RelocLgSetOrBySection" (LSEC) instruction is a group including the
// "RelocLgBySection",
// "RelocLgSetSectC"
// "RelocLgSetSectD"
bitfield RelocLgSetOrBySection
{
op : 6; // 101101
subOp : 4;
index : 22;
};
enum relocLgSetOrBySectionAdditionalOpcode :u8
{
kPEFRelocLgBySectionSubopcode = 0x00, /* Binary: 0000*/
kPEFRelocLgSetSectCSubopcode = 0x01, /* Binary: 0001*/
kPEFRelocLgSetSectDSubopcode = 0x02 /* Binary: 0010*/
};
struct RelocInstruction
{
u8 firstByte[[no_unique_address, hidden]];
if (firstByte & 0b11000000 == 0)
{
RelocBySectDWithSkip instruction;
std::core::set_display_name(this, "RelocBySectDWithSkip");
}
else if (firstByte & 0b01000000 == 0x40)
{
RelocRunGroup instruction;
if (instruction.subOp == 0)
{
std::core::set_display_name(this, "RelocBySectC");
}
else if (instruction.subOp == 1)
{
std::core::set_display_name(this, "RelocBySectD");
}
else if (instruction.subOp == 2)
{
std::core::set_display_name(this, "RelocTVector12");
}
else if (instruction.subOp == 3)
{
std::core::set_display_name(this, "RelocTVector8");
}
else if (instruction.subOp == 4)
{
std::core::set_display_name(this, "RelocVTable8");
}
else if (instruction.subOp == 5)
{
std::core::set_display_name(this, "RelocImportRun");
}
}
else if (firstByte & 0b01100000 == 0x60)
{
RelocSmIndexGroup instruction;
if (instruction.subOp == 0)
{
std::core::set_display_name(this, "RelocSmByImport");
}
else if (instruction.subOp == 1)
{
std::core::set_display_name(this, "RelocSmSetSectC");
}
else if (instruction.subOp == 2)
{
std::core::set_display_name(this, "RelocSmSetSectD");
}
else if (instruction.subOp == 3)
{
std::core::set_display_name(this, "RelocSmBySection");
}
}
else if (firstByte & 0b11110000 == relocationOpcode::kPEFRelocIncrPosition << 1)
{
RelocIncrPosition instruction;
std::core::set_display_name(this, "RelocIncrPosition");
}
else if (firstByte & 0b11110000 == relocationOpcode::kPEFRelocSmRepeat << 1)
{
RelocSmRepeat instruction;
std::core::set_display_name(this, "RelocSmRepeat");
}
else if (firstByte & 0b11111100 == relocationOpcode::kPEFRelocSetPosition << 1)
{
RelocSetPosition instruction;
std::core::set_display_name(this, "RelocSetPosition");
}
else if (firstByte & 0b11111100 == relocationOpcode::kPEFRelocLgByImport << 1)
{
RelocLgByImport instruction;
std::core::set_display_name(this, "RelocLgByImport");
}
else if (firstByte & 0b11111100 == relocationOpcode::kPEFRelocLgRepeat << 1)
{
RelocLgRepeat instruction;
std::core::set_display_name(this, "RelocLgRepeat");
}
else if (firstByte & 0b11111100 == relocationOpcode::kPEFRelocLgSetOrBySection << 1)
{
RelocLgSetOrBySection instruction;
if (instruction.subOp == kPEFRelocLgBySectionSubopcode)
{
std::core::set_display_name(this, "RelocLgBySectionSubopcode");
}
else if (instruction.subOp == kPEFRelocLgSetSectCSubopcode)
{
std::core::set_display_name(this, "RelocLgSetSectCSubopcode");
}
else if (instruction.subOp == kPEFRelocLgSetSectDSubopcode)
{
std::core::set_display_name(this, "RelocLgSetSectDSubopcode");
}
}
};
bitfield PEFExportedSymbolHashSlot
{
count : 14;
start : 18;
};
struct PEFExportedSymbolKey
{
u16 nameLength;
u16 hashValue;
};
enum exportSymbolClass :u8
{
kPEFAbsoluteExport = -2, /* The symbol value is an absolute address.*/
kPEFReexportedImport = -3 /* The symbol value is the index of a reexported import.*/
};
struct PEFExportSymbol
{
exportSymbolClass class [[comment("Indicates the class of the exported symbol.")]];
u24 nameOffset [[comment("Indicates the offset (in bytes) from the beginning of the loader string table to the null-terminated name of the symbol")]];
};
struct PEFExportedSymbol
{
PEFExportSymbol classAndName [[comment("The first byte designates the symbol class of the exported symbol. See \"symbolClass\" enum. Flag bits for exported symbols are reserved for future use. The following 3 bytes designate the offset from the beginning of the loader string table to the name of the symbol. The name of the symbol is not null terminated, but you can determine the length of the string from the upper 2 bytes of the symbols hash word (found in the export key table).")]];
u32 symbolValue [[comment("Typically indicates the offset from the beginning of the symbols section to the exported symbol.")]];
s16 sectionIndex [[comment("Indicates the number of the section that contains this symbol. Note that this is a signed field.")]];
};
PEFContainerHeader containerHeader @ 0x00;
PEFSectionHeader sectionsTable[containerHeader.sectionCount] @ sizeof(containerHeader);
fn getFirstSectionInContainer()
{
u32 minOff = 0xFFFFFFFF;
for (u32 i = 0, i < containerHeader.sectionCount, i += 1)
{
u32 sectionOffset = sectionsTable[i].containerOffset;
if (sectionOffset < minOff)
{
minOff = sectionOffset;
}
}
return minOff;
};
u32 firstSectionOffset = getFirstSectionInContainer();
u32 sectionNameTableBase = sizeof(containerHeader) + sizeof(sectionsTable);
String sectionNameTable[while($ < firstSectionOffset)] @ sectionNameTableBase;
fn getLoaderSection()
{
for (u32 i = 0, i < containerHeader.sectionCount, i += 1)
{
if (sectionsTable[i].sectionKind == SectionKind::kPEFLoaderSection)
{
return sectionsTable[i].containerOffset;
}
}
};
u32 loaderOffset = getLoaderSection();
PEFLoaderInfoHeader loaderHeader @ loaderOffset;
u32 loaderStringTableOffset = loaderOffset + loaderHeader.loaderStringsOffset;
String loaderStringTable[while($ < loaderOffset + loaderHeader.exportHashOffset)] @ loaderStringTableOffset;
fn format_name(auto pat)
{
u32 i = 0;
u32 nameAddress = loaderStringTableOffset + pat.nameOffset;
String string @ nameAddress;
return string;
};
PEFImportedLibrary importTableHeaders[loaderHeader.importedLibraryCount] @ loaderOffset + sizeof(loaderHeader);
u32 importSymbolsOffset = loaderOffset + sizeof(loaderHeader) + loaderHeader.importedLibraryCount * sizeof(PEFImportedLibrary);
PEFImportSymbol importSymbols[loaderHeader.totalImportedSymbolCount] @ importSymbolsOffset;
u32 relocationHeaderOffset = importSymbolsOffset + sizeof(importSymbols);
PEFLoaderRelocationHeader relocationHeaders[loaderHeader.relocSectionCount] @ relocationHeaderOffset;
u32 relocBegin = loaderOffset + loaderHeader.relocInstrOffset;
u32 relocSize = loaderOffset + loaderHeader.loaderStringsOffset - relocBegin;
RelocInstruction relocationInstructions[while(($-relocBegin) < relocSize)] @ relocBegin;
u32 exportSlotNum = std::math::pow(2, loaderHeader.exportHashTablePower);
u32 exportDataOffset = loaderOffset + loaderHeader.exportHashOffset;
u32 exportSymbolKeysOffset = exportDataOffset + sizeof(PEFExportedSymbolHashSlot) * exportSlotNum;
u32 exportedSymbolsOffset = exportSymbolKeysOffset + sizeof(PEFExportedSymbolKey) * loaderHeader.exportedSymbolCount;
PEFExportedSymbolHashSlot exportSlots[exportSlotNum] @ exportDataOffset;
PEFExportedSymbolKey exportSymbolKeys[loaderHeader.exportedSymbolCount] @ exportSymbolKeysOffset;
PEFExportedSymbol exportedSymbols[loaderHeader.exportedSymbolCount] @ exportedSymbolsOffset;

542
patterns/pex.hexpat Normal file
View File

@@ -0,0 +1,542 @@
#pragma author Scrivener07 and Jonathan Ostrus
#pragma description Bethesda Papyrus executable compiled script file
#pragma magic [FA 57 C0 DE] @ 0x00
#pragma magic [DE C0 57 FA] @ 0x00
import std.sys;
import std.core;
import std.string;
import std.time;
enum GAMEID : u16 {
GAME_Skyrim = 1,
GAME_Fallout4 = 2,
GAME_Fallout76 = 3,
GAME_Starfield = 4
};
GAMEID g_GameId;
namespace formatter {
fn pexversion(ref auto data) {
return std::format("{}.{}", data.MajorVersion, data.MinorVersion);
};
fn structname(ref auto data) {
return StringLookup(data.name);
};
fn typedstructname(ref auto data) {
return std::format("{0} {1}", StringLookup(data.typeName), StringLookup(data.name));
};
fn time(ref auto data) {
// We will only support dates back to the epoch
std::time::Time time64 = std::time::to_utc(data.Time);
return std::time::format(time64, "%c");
};
fn vartype(u8 data) {
match (data)
{
(0x0): return "object";
(0x1): return "identifier";
(0x2): return "string";
(0x3): return "integer";
(0x4): return "float";
(0x5): return "bool";
(_): return "Unknown type";
}
};
fn vartypestruct(ref auto data) {
return formatter::vartype(data.varType);
};
fn arraycount(ref auto data) {
return std::format("[{}]", data.count);
};
fn functiontype(u8 data) {
match(data) {
(0): return "regular";
(1): return "property getter";
(2): return "property setter";
(3): std::assert(true, "Unknown function type");
}
};
fn resolveop(u8 op) {
match (op)
{
(0x00): return "nop";
(0x01): return "iadd";
(0x02): return "fadd";
(0x03): return "isub";
(0x04): return "fsub";
(0x05): return "imul";
(0x06): return "fmul";
(0x07): return "idiv";
(0x08): return "fdiv";
(0x09): return "imod";
(0x0A): return "not";
(0x0B): return "ineg";
(0x0C): return "fneg";
(0x0D): return "assign";
(0x0E): return "cast";
(0x0F): return "cmp_eq";
(0x10): return "cmp_lt";
(0x11): return "cmp_lte";
(0x12): return "cmp_gt";
(0x13): return "comp_gte";
(0x14): return "jmp";
(0x15): return "jmpt";
(0x16): return "jmpf";
(0x17): return "callmethod";
(0x18): return "callparent";
(0x19): return "callstatic";
(0x1A): return "return";
(0x1B): return "strcat";
(0x1C): return "propget";
(0x1D): return "propset";
(0x1E): return "array_create";
(0x1F): return "array_length";
(0x20): return "array_getlement";
(0x21): return "array_setelement";
(0x22): return "array_findelement";
(0x23): return "array_rfindelement";
(0x24): return "is";
(0x25): return "struct_create";
(0x26): return "struct_get";
(0x27): return "struct_set";
(0x28): return "array_findstruct";
(0x29): return "array_rfindstruct";
(0x2A): return "array_add";
(0x2B): return "array_insert";
(0x2C): return "array_removelast";
(0x2D): return "array_remove";
(0x2E): return "array_clear";
(0x2F): return "array_getallmatchingstructs";
(0x30): return "lock_guards";
(0x31): return "unlock_guards";
(0x32): return "try_lock_guards";
(_): return "Unknown";
}
};
fn resolveopstruct(auto data) {
return formatter::resolveop(data.op);
};
}
fn StringLookup(u16 idx) {
return pex.stringsTable.strings[idx];
};
struct PexVersion {
u8 MajorVersion;
u8 MinorVersion;
} [[sealed, format("formatter::pexversion")]];
using StringReference = u16 [[format("StringLookup")]];
// Dynamic array of strings
struct StringsTable {
u16 count;
if (count > 0)
std::string::SizedString<u16> strings[count];
} [[format("formatter::arraycount")]];
struct Time {
u64 Time;
} [[sealed, format("formatter::time")]];
struct VariableData {
u8 varType [[format("formatter::vartype")]];
match (varType)
{
(0x0): {} // object pointer
(0x1): StringReference stringVal; // identifier
(0x2): StringReference stringVal; // string
(0x3): s32 intVal; // integer
(0x4): float floatVal;
(0x5): bool boolVal;
(_): std::assert(false, "Unknown type for variable data");
}
} [[format("formatter::vartypestruct")]];
struct VariableType {
StringReference name;
StringReference varType;
};
struct VariableTypes {
u16 count;
if (count > 0)
VariableType parameter[count] [[inline]];
} [[format("formatter::arraycount")]];
struct Instruction {
u8 op [[format("formatter::resolveop")]];
match (op)
{
(
0x00 // nop
): {}
(
0x01 | // iadd
0x02 | // fadd
0x03 | // isub
0x04 | // fsub
0x05 | // imul
0x06 | // fmul
0x07 | // idiv
0x08 | // fdiv
0x09 | // imod
0x0F | // cmp_eq
0x10 | // cmp_lt
0x11 | // cmp_lte
0x12 | // cmp_gt
0x13 | // comp_gte
0x1B | // strcat
0x1C | // propget
0x1D | // propset
0x20 | // array_getlement
0x21 | // array_setelement
0x24 | // is
0x26 | // struct_get
0x27 | // struct_set
0x2A | // array_add
0x2B | // array_insert
0x2D // array_remove
): VariableData arguments[3];
(
0x22 | // array_findelement
0x23 // array_rfindelement
): VariableData arguments[4];
(
0x14 | // jmp
0x1A | // return
0x25 | // struct_create
0x2C | // array_removelast
0x2E // array_clear
): VariableData arguments[1];
(
0x0A | // not
0x0B | // ineg
0x0C | // fneg
0x0D | // assign
0x0E | // cast
0x15 | // jmpt
0x16 | // jmpf
0x1E | // array_create
0x1F // array_length
): VariableData arguments[2];
(
0x28 | // array_findstruct
0x29 // array_rfindstruct
): VariableData arguments[5];
(
0x2F // array_getallmatchingstructs
): VariableData arguments[6];
(
0x17 | // callmethod
0x19 // callstatic
): {
VariableData arguments[4];
std::assert(arguments[3].varType == 0x3, "VarArgs not integer");
if (arguments[3].intVal > 0)
VariableData vararguments[arguments[3].intVal];
}
(
0x18 // callparent
): {
VariableData arguments[3];
std::assert(arguments[2].varType == 0x3, "VarArgs not integer");
if (arguments[2].intVal > 0)
VariableData vararguments[arguments[2].intVal];
}
(
0x30 | // lock_guards
0x31 // unlock_guards
): {
VariableData arguments[1];
std::assert(arguments[0].varType == 0x3, "VarArgs not integer");
if (arguments[0].intVal > 0)
VariableData vararguments[arguments[0].intVal];
}
(
0x32 // try_lock_guards
): {
VariableData arguments[2];
std::assert(arguments[1].varType == 0x3, "VarArgs not integer");
if (arguments[1].intVal > 0)
VariableData vararguments[arguments[1].intVal];
}
(_): {
std::print("Invalid opcode: {:02x}", op);
std::assert(false, "Invalid opcode");
}
}
} [[format("formatter::resolveopstruct")]];;
struct Instructions {
u16 count;
if (count > 0)
Instruction instruction[count] [[inline]];
} [[format("formatter::arraycount")]];
/*
this "BaseFunction" struct is used by the Papyrus struct property
getter and setter functions, and by regular named functions.
*/
struct BaseFunction {
StringReference returnType;
StringReference docString;
u32 userFlags;
u8 flags;
VariableTypes arguments;
VariableTypes locals;
Instructions instructions;
};
struct NamedFunction {
StringReference name;
BaseFunction function [[inline]];
} [[format("formatter::structname")]];
struct NamedFunctions {
u16 count;
if (count > 0)
NamedFunction function[count] [[inline]];
} [[format("formatter::arraycount")]];
struct State {
StringReference name;
NamedFunctions functions;
} [[format("formatter::structname")]];
struct SyncStates {
u16 count;
if (count > 0)
StringReference names[count] [[inline]];
} [[format("formatter::arraycount")]];
struct States {
u16 count;
if (count > 0)
State state[count] [[inline]];
} [[format("formatter::arraycount")]];
struct Property {
StringReference name;
StringReference typeName;
StringReference docString;
u32 userFlags;
u8 flags;
if ((flags & 0x4) != 0)
StringReference autoVarName;
else {
if ((flags & 0x1) != 0)
BaseFunction readHandler;
if ((flags & 0x2) != 0)
BaseFunction writeHandler;
}
} [[format("formatter::structname")]];
struct Properties {
u16 count;
if (count > 0)
Property property[count] [[inline]];
} [[format("formatter::arraycount")]];
struct Variable {
StringReference name;
StringReference typeName;
u32 userFlags;
VariableData data;
if (g_GameId >= GAMEID::GAME_Fallout4)
u8 constFlag;
} [[format("formatter::structname")]];;
struct Variables {
u16 count;
if (count > 0)
Variable variable[count] [[inline]];
} [[format("formatter::arraycount")]];
struct ObjectStructMember {
StringReference name;
StringReference typeName;
u32 userFlags;
VariableData data;
u8 constFlag;
StringReference docString;
} [[format("formatter::typedstructname")]];
struct ObjectStructMembers {
u16 count;
if (count > 0)
ObjectStructMember members[count] [[inline]];
} [[format("formatter::arraycount")]];
struct ObjectStruct {
StringReference name;
ObjectStructMembers members;
} [[format("formatter::structname")]];
struct ObjectStructs {
u16 count;
if (count > 0)
ObjectStruct structs[count] [[inline]];
} [[format("formatter::arraycount")]];
struct Guards {
u16 count;
if (count > 0)
StringReference names[count] [[inline]];
} [[format("formatter::arraycount")]];
struct ScriptObjectData {
StringReference parentClassName;
StringReference docString;
if (g_GameId >= GAMEID::GAME_Fallout4)
u8 constFlag;
u32 userFlags;
StringReference autoStateName;
if (g_GameId >= GAMEID::GAME_Fallout4)
ObjectStructs structs;
Variables variables;
if (g_GameId >= GAMEID::GAME_Starfield)
Guards guards;
Properties properties;
States states;
if (g_GameId == GAMEID::GAME_Fallout76)
SyncStates syncStates;
};
struct ScriptObject {
StringReference name;
u32 length;
ScriptObjectData data;
// The BGS Compiler sets length to sizeof(data) + sizeof(length)
// The Caprica compiler sets the length to just sizeof(data)
// Since we have no way to identify which compiler is used we'll check
// both and just hope it isn't short exactly 4 bytes
std::assert(sizeof(data) == length - 4 || sizeof(data) == length, "Length of object data in state missmatch.");
};
struct ScriptObjects {
u16 count;
if (count > 0)
ScriptObject scriptObjects[count] [[inline]];
} [[format("formatter::arraycount")]];
struct UserFlag {
StringReference name;
u8 flagIndex;
} [[format("formatter::structname")]];
struct UserFlags {
u16 count;
if (count > 0)
UserFlag userFlags[count] [[inline]];
} [[format("formatter::arraycount")]];
struct StringMembers {
u16 count;
if (count > 0)
StringReference members[count] [[inline]];
} [[format("formatter::arraycount")]];
struct DebugPropertyGroup {
StringReference objectName;
StringReference name;
StringReference docString;
u32 userFlags;
StringMembers members;
} [[format("formatter::structname")]];
struct DebugInstructions {
u16 count;
if (count > 0)
u16 lineNumbers[count] [[inline]];
} [[format("formatter::arraycount")]];
struct DebugFunction {
StringReference objectName;
StringReference stateName;
StringReference name;
u8 functionType [[format("formatter::functiontype")]];
DebugInstructions instructions;
} [[format("formatter::structname")]];
struct DebugStruct {
StringReference parentName;
StringReference name;
StringMembers members;
} [[format("formatter::structname")]];
struct DebugFunctions {
u16 count;
if (count > 0)
DebugFunction functions[count] [[inline]];
} [[format("formatter::arraycount")]];
struct DebugPropertyGroups {
u16 count;
if (count > 0)
DebugPropertyGroup groups[count] [[inline]];
} [[format("formatter::arraycount")]];
struct DebugStructs {
u16 count;
if (count > 0)
DebugStruct structs[count] [[inline]];
} [[format("formatter::arraycount")]];
struct DebugInfo {
bool hasDebugInfo;
if (hasDebugInfo)
{
Time modificationTime;
DebugFunctions functions;
if (g_GameId >= GAMEID::GAME_Fallout4)
{
DebugPropertyGroups groups;
DebugStructs structs;
}
}
};
// Header
struct Header {
u32 Magic;
PexVersion pexVersion;
GAMEID GameID; g_GameId = GameID;
Time CompilationTime;
std::string::SizedString<u16> SourceFileName;
std::string::SizedString<u16> UserName;
std::string::SizedString<u16> MachineName;
};
struct PEX {
if (std::mem::read_unsigned(0, 4) == le u32(0xDEC057FA))
std::core::set_endian(std::mem::Endian::Big);
else
std::core::set_endian(std::mem::Endian::Little);
Header header;
StringsTable stringsTable;
DebugInfo debugInfo;
UserFlags userFlags;
ScriptObjects scriptObjects;
};
PEX pex @ 0x00;

161
patterns/pf.hexpat Normal file
View File

@@ -0,0 +1,161 @@
#pragma author MrMcX
#pragma description Parse forensically relevant parts of Windows Prefetch(*.pf) (only uncompressed SCCA version)
/*
The pattern implements the uncompressed SCCA format which is used below Windows 10.
For the newer compressed file format starting with "MAM\x04" has to be decompressed first,
for example using the at https://gist.github.com/dfirfpi/113ff71274a97b489dfd
Thanks to Joachim Metz for his work that this pattern is heavily based on:
https://github.com/libyal/libscca/blob/main/documentation/Windows%20Prefetch%20File%20(PF)%20format.asciidoc
Unknown values commented with "jm:" refer to Joachim Metz' format analysis
*/
#pragma magic [0x54 0x43 0x43 0x41] @ 0x04
#pragma endian little
import std.mem;
import std.string;
import type.time;
import type.magic;
enum WinVer : u32 {
Windows11 = 31,
Windows10 = 30,
Windows8x = 26,
Windows7orVista = 23,
WindowsXPor2003 = 17
};
enum Flag : u32 {
Boot = 0x01,
Application =0x00
};
fn to_HexString32(u32 value) {
return std::string::to_upper(std::format("{:08x}", value));
};
using HexString32 = u32 [[format("to_HexString32")]];
struct Header {
WinVer version;
type::Magic<"SCCA"> signature;
u32;
u32 size;
char16 program_name[30];
HexString32 crc_hash;
Flag flag;
};
struct FileInformation {
u32 metrics_offset;
u32 metrics_entry_count;
u32 trace_chains_offset;
u32 trace_chains_entry_count;
u32 filenames_offset;
u32 filenames_size;
u32 volumes_information_offset;
u32 volumes_count;
u32 volumes_information_size;
if(header.version >= WinVer::Windows7orVista) {
padding[8];
}
// Times are in FILETIME format (now shown as unix timestamp)
type::FILETIME last_execution;
if(header.version >= WinVer::Windows8x) {
type::FILETIME prev_executions[7];
}
if(header.version <= WinVer::Windows10) {
u128;
} else {
u64;
}
u32 run_count;
u32;
u32 hash_string_offset = 0x0;
u32 hash_string_size = 0;
if(header.version >= WinVer::Windows10) {
u32;
u32 hash_string_offset;
u32 hash_string_size;
padding[76];
} else if(header.version == WinVer::Windows8x) {
u32;
padding[84];
} else if(header.version == WinVer::Windows7orVista) {
padding[80];
}
};
struct FileReference {
u48 mft_entry_index;
u16 sequence_number;
};
struct FileMetric {
u32; // jm: might be prefetch start time in ms or index into the trace chain array
u32; // jm: might be prefetch duration in ms or number of entries in the trace chain
if(header.version >= WinVer::Windows7orVista) {
u32; // jm: might be Average prefetch duration in ms?
}
u32 filename_string_offset;
u32 filename_string_length;
u32 flags;
if(header.version >= WinVer::Windows7orVista) {
FileReference file_reference;
}
};
struct TraceChain {
if(header.version <= WinVer::Windows8x) {
u32 next_array_entry_index;
}
u32 total_block_load_count;
u8;
u8; // jm: might be sample duration in ms
u16;
};
struct DirectoryString {
u16 length;
char16 value[length+1];
};
struct VolumeInformation {
auto vi_start = $;
u32 volume_device_path_offset;
u32 volume_device_path_length;
type::FILETIME volume_creation_time;
HexString32 volume_serial_number;
u32 file_references_offset;
u32 file_references_size;
u32 directory_strings_offset;
u32 directory_strings_count;
u32;
if(header.version >= WinVer::Windows7orVista) {
padding[(header.version >= WinVer::Windows10) ? 24 : 28];
u32; // jm: might be copy of the number of directory strings
padding[(header.version >= WinVer::Windows10) ? 24 : 28];
u32;
}
$ = vi_start + volume_device_path_offset;
char16 volume_device_path[volume_device_path_length];
$ = vi_start + file_references_offset;
u32 file_reference_version;
u32 file_reference_count;
if(header.version >= WinVer::Windows7orVista) {
u64;
}
FileReference file_references[file_reference_count];
$ = vi_start + directory_strings_offset;
DirectoryString directory_strings[directory_strings_count];
};
Header header @ 0x00;
FileInformation info @ 0x54;
FileMetric metrics[info.metrics_entry_count] @ info.metrics_offset;
TraceChain trace_chains[info.trace_chains_entry_count] @ info.trace_chains_offset;
std::string::NullString16 filenames[while(!std::mem::reached(info.hash_string_offset))] @ info.filenames_offset;
// executable_path should be empty below windows 10
char16 executable_path[(info.hash_string_size / 2) - 1] @ info.hash_string_offset;
VolumeInformation volume_informations[info.volumes_count] @ info.volumes_information_offset;

View File

@@ -1,12 +1,25 @@
#pragma author WerWolv
#pragma author WerWolv and Glenn Hartmann
#pragma description Google Protobuf wire encoding (.pb)
#pragma MIME application/protobuf
#pragma MIME application/vnd.google.protobuf
#pragma endian little
import std.core;
import std.io;
import std.mem;
import std.string;
import std.sys;
import type.leb128;
// Attempting to recursively parse submessages is a guess-and-check process
// since it's inherently impossible to tell for sure what type a
// LengthDelimited field is. This could be imprecise and could be slow for
// large or ambiguous files, so we give the user an option to disable it.
bool disable_recursive_submessage_parsing in;
struct ZigZag32 {
u32 value;
} [[sealed, format("format_zigzag32")]];
@@ -32,7 +45,6 @@ enum WireType : u8 {
_32Bit = 5
};
struct Key {
type::uLEB128 keyDec;
u32 field_number = u32(keyDec) >> 3;
@@ -55,23 +67,55 @@ union _32Bit {
float flt;
};
using Field;
struct Message<auto Size> {
Field fields[while(!std::mem::reached(addressof(this) + Size))];
};
struct Utf8String<auto Length> {
char data[Length];
} [[sealed, format("std::string::impl::format_string"), transform("std::string::impl::format_string")]];
union _LengthDelimitedData<auto Length> {
u8 bytes[Length];
Utf8String<Length> utf8;
if (!disable_recursive_submessage_parsing) {
try {
// Attempt to parse binary data as an embedded Message. This is
// expected to fail often, as the proto format uses LengthDelimited
// for several different data types.
Message<Length> msg;
std::assert(sizeof(msg) == Length, "Attempted parse of Message consumed wrong number of bytes.");
}
}
};
struct LengthDelimited {
type::uLEB128 length;
char data[length];
std::assert($ + length <= std::mem::size(), "Attempting to parse _LengthDelimitedData would exceed file length.");
_LengthDelimitedData<length> data;
};
union _LEB128 {
type::uLEB128 uLEB128;
type::sLEB128 sLEB128; // NOTE: the signed version doesn't seem to be working properly
};
struct Entry {
struct Field {
Key key;
if (key.wire_type == WireType::Varint)
type::uLEB128 value;
else if (key.wire_type == WireType::_64Bit)
_64Bit value;
else if (key.wire_type == WireType::LengthDelimited)
LengthDelimited value;
else if (key.wire_type == WireType::_32Bit)
_32Bit value;
match (key.wire_type) {
(WireType::Varint): _LEB128 value;
(WireType::_64Bit): _64Bit value;
(WireType::LengthDelimited): LengthDelimited value;
(WireType::_32Bit): _32Bit value;
(WireType::StartGroup | WireType::EndGroup): std::unimplemented();
(_): std::error("Unknown WireType.");
}
};
Entry entries[while(!std::mem::eof())] @ 0x00;
Message<std::mem::size()> msg @ 0x00;
std::assert(std::mem::eof(), "Parsing did not consume whole file.");

View File

@@ -277,6 +277,8 @@ enum Magic : u32 {
MAGIC_3_9 = 0x0A0D0D61,
MAGIC_3_10 = 0x0A0D0D6F,
MAGIC_3_11 = 0x0A0D0DA7,
MAGIC_3_12 = 0x0A0D0DCB,
MAGIC_3_13 = 0x0A0D0DF3,
INVALID = 0,
};
@@ -285,7 +287,7 @@ fn getMajor(Magic magic) {
match(magic) {
(Magic::MAGIC_1_0 | Magic::MAGIC_1_1 | Magic::MAGIC_1_3 | Magic::MAGIC_1_4 | Magic::MAGIC_1_5 | Magic::MAGIC_1_6): return 1;
(Magic::MAGIC_2_0 | Magic::MAGIC_2_1 | Magic::MAGIC_2_2 | Magic::MAGIC_2_3 | Magic::MAGIC_2_4 | Magic::MAGIC_2_5 | Magic::MAGIC_2_6 | Magic::MAGIC_2_7): return 2;
(Magic::MAGIC_3_0 | Magic::MAGIC_3_1 | Magic::MAGIC_3_2 | Magic::MAGIC_3_3 | Magic::MAGIC_3_4 | Magic::MAGIC_3_5 | Magic::MAGIC_3_5_3 | Magic::MAGIC_3_6 | Magic::MAGIC_3_7 | Magic::MAGIC_3_8 | Magic::MAGIC_3_9 | Magic::MAGIC_3_10 | Magic::MAGIC_3_11): return 3;
(Magic::MAGIC_3_0 | Magic::MAGIC_3_1 | Magic::MAGIC_3_2 | Magic::MAGIC_3_3 | Magic::MAGIC_3_4 | Magic::MAGIC_3_5 | Magic::MAGIC_3_5_3 | Magic::MAGIC_3_6 | Magic::MAGIC_3_7 | Magic::MAGIC_3_8 | Magic::MAGIC_3_9 | Magic::MAGIC_3_10 | Magic::MAGIC_3_11 | Magic::MAGIC_3_12 | Magic::MAGIC_3_13): return 3;
(Magic::INVALID): return 0;
}
};
@@ -304,6 +306,8 @@ fn getMinor(Magic magic) {
(Magic::MAGIC_3_9): return 9;
(Magic::MAGIC_3_10): return 10;
(Magic::MAGIC_3_11): return 11;
(Magic::MAGIC_3_12): return 12;
(Magic::MAGIC_3_13): return 13;
(Magic::INVALID): return 0;
}
};

201
patterns/q3demo.hexpat Normal file

File diff suppressed because one or more lines are too long

49
patterns/rcf_v1_2.hexpat Normal file
View File

@@ -0,0 +1,49 @@
// Ported from LRCFB @ https://donutteam.com/@lucasc190/releases
// Tested with files from:
// - Hulk (PS2)
// - Tetris Worlds (Gamecube)
// - The Simpsons Hit and Run (PC)
#pragma author blu
#pragma description Radcore Cement Library file header (.rcf) v1.2
import std.core;
import type.time;
struct DataEntry {
u32 hash;
u32 position;
u32 size;
};
struct NameEntry {
le u32 length;
char name[length];
le type::time32_t time;
};
struct Header {
char signature[32];
u8 version_major;
u8 version_minor;
bool is_bigendian; // true for gamecube
u8 unk_0; //always "1" according to LRCFB
if (is_bigendian) {
std::core::set_endian(std::mem::Endian::Big);
}
u32 alignment;
u32 unk_1;
u32 directory_position;
$ = directory_position;
s32 data_count;
u32 names_position;
u32 data_position;
le u32 data_pointer;
DataEntry data_entries[data_count];
$ = names_position;
le u32 names_count;
le u32 names_pointer;
NameEntry name_entries[names_count];
};
Header header @ 0x00;

View File

@@ -0,0 +1,33 @@
#pragma author marduk.ru
#pragma description Roblox .pack shader archive format
#pragma magic [ 52 42 58 53 ] @ 0x00
import std.string;
import hex.core;
import type.magic;
struct Header {
type::Magic<"RBXS"> magic;
u32 fileCount;
};
struct FileIndex {
char name[0x40];
u128 md5Hash;
u32 offset;
u32 size;
u64 reserved;
u8 file[size] @ offset;
#ifdef __IMHEX__
hex::core::add_virtual_file(name, file);
#endif
} [[name(name)]];
struct RBXPack {
Header header;
FileIndex files[header.fileCount];
};
RBXPack pack @ 0x0;

272
patterns/rpm.hexpat Normal file
View File

@@ -0,0 +1,272 @@
#pragma author k0tran
#pragma description RPM package format
#pragma endian big
#pragma magic [ ED AB EE DB ] @ 0x00
import std.mem;
import std.core;
import std.string;
enum Magic : u32 {
magic = 0xEDABEEDB
};
enum PackageType : s16 {
binary = 0x0000,
source = 0x0001
};
struct Lead {
Magic magic [[hidden]];
u8 major [[comment("Version major")]];
u8 minor [[comment("Version minor")]];
PackageType type [[comment("Binary/Source")]];
s16 archnum;
char name[66] [[comment("Package name")]];
s16 osnum;
s16 signature_type;
char reserved[16] [[hidden]];
};
enum HeaderMagic : u24 {
magic = 0x8EADE8
};
enum Tag : u32 {
// private
headerimage = 61,
headersignatures = 62,
headerimmutable = 63,
headerregions = 64,
headero18ntable = 100,
sig_base = 256,
sigsize = 257,
siglemd5_1 = 258,
sigpgp = 259,
siglemd5_2 = 260,
sigmd5 = 261, // or pkgid
siggpg = 262,
sigpgp5 = 263,
badsha1_1 = 264,
badsha1_2 = 265,
pubkeys = 266,
dsaheader = 267,
rsaheader = 268,
sha1header = 269, // or hdrid
longsigsize = 270,
longarchivesize = 271,
// 272 reserved
sha256header = 273,
// 274, 275 reserved
veritysignatures = 276,
veritysignaturealgo = 277,
openpgp = 278,
sig_top = 279,
// public
name = 1000,
version = 1001,
release = 1002,
epoch = 1003, // or serial
summary = 1004,
description = 1005,
buildtime = 1006,
buildhost = 1007,
installtime = 1008,
size = 1009,
distribution = 1010,
vendor = 1011,
gif = 1012,
xpm = 1013,
license = 1014, // or copyright
packager = 1015,
group = 1016,
changelog = 1017,
source = 1018,
patch = 1019,
url = 1020,
os = 1021,
arch = 1022,
prein = 1023,
postin = 1024,
preun = 1025,
postun = 1026,
oldfilenames = 1027,
filesizes = 1028,
filestates = 1029,
filemdes = 1030,
fileuids = 1031,
filegids = 1032,
filerdevs = 1033,
filemtimes = 1034,
filemd5s = 1035,
filelinktos = 1036,
fileflags = 1037,
root = 1038,
fileusername = 1039,
filegroupname = 1040,
exclude = 1041,
exclusive = 1042,
icon = 1043,
sourcerpm = 1044,
fileverifyflags = 1045,
archivesize = 1046,
providename = 1047, // or provides
requireflags = 1048,
requirename = 1049,
requireversion = 1050,
nosource = 1051,
nopatch = 1052,
conflictflags = 1053,
conflictname = 1054,
conflictversion = 1055,
defaultprefix = 1056,
buildroot = 1057,
installprefix = 1058,
excludearch = 1059,
excludeos = 1060,
exclusivearch = 1061,
exclusiveos = 1062,
autoreqprov = 1063,
rpmversion = 1064,
triggerscripts = 1065,
triggername = 1066,
triggerversion = 1067,
triggerflags = 1068,
triggerindex = 1069,
// reserved
verifyscript = 1079,
changelogtime = 1080,
changelogname = 1081,
changelogtext = 1082,
brokenmd5 = 1083,
prereq = 1084,
preinprog = 1085,
postinprog = 1086,
preunprog = 1087,
postunprog = 1088,
buildarchs = 1089,
obsoletename = 1090, // or obsoletes
verifyscriptprog = 1091,
triggerscriptprog = 1092,
docdir = 1093,
cookie = 1094,
filedevices = 1095,
fileinodes = 1096,
filelangs = 1097,
prefixes = 1098,
instprefixes = 1099,
triggerin = 1100,
triggerun = 1101,
triggerpostun = 1102,
autoreq = 1103,
autoprov = 1104,
capability = 1105,
sourcepackage = 1106,
oldorigfilenames = 1107,
buildprereq = 1108,
buildrequires = 1109,
buildconflicts = 1110,
buildmacros = 1111,
provideflags = 1112,
provideversion = 1113,
obsoleteflags = 1114,
obsoleteversion = 1115,
dirindexes = 1116,
basenames = 1117,
dirnames = 1118,
origdirindexes = 1119,
origbasenames = 1120,
origdirnames = 1121,
optflags = 1122,
disturl = 1123,
payloadformat = 1124,
payloadcompressor = 1125,
payloadflags = 1126,
installcolor = 1127,
installtid = 1128,
removetid = 1129,
sha1rhn = 1130,
rhnplatform = 1131,
platform = 1132,
patchesname = 1133,
patchesflags = 1134,
patchesversion = 1135,
cachectime = 1136,
cachepkgpath = 1137,
cachepkgsize = 1138,
cachepkgmtime = 1139,
filecolors = 1140,
fileclass = 1141,
classdict = 1142,
filedependsx = 1143,
filedependsn = 1144,
dependsdict = 1145,
sourcepkgid = 1146,
filecontexts = 1147,
fscontexts = 1148,
recontexts = 1149,
policies = 1150,
};
enum IndexType : u32 {
_null = 0,
_char = 1,
_int8 = 2,
_int16 = 3,
_int32 = 4,
_int64 = 5,
_string = 6,
_bin = 7,
_string_array = 8,
};
struct Index {
Tag tag;
IndexType type;
u32 offset [[comment("Store offset")]];
u32 count [[comment("N of that datatype")]];
};
struct String {
char string_array_item[];
} [[sealed]];
struct StoreEntry {
auto i = std::core::array_index();
auto p = parent.offset + parent.index[i].offset;
auto c = parent.index[i].count;
str info = std::format("{:X} {}", parent.index[i].offset, parent.index[i].tag);
match (parent.index[i].type) {
(IndexType::_char): char store_entry[c] @ p [[sealed, comment(info)]];
(IndexType::_int8): s8 store_entry[c] @ p [[sealed, comment(info)]];
(IndexType::_int16): s16 store_entry[c] @ p [[sealed, comment(info)]];
(IndexType::_int32): s32 store_entry[c] @ p [[sealed, comment(info)]];
(IndexType::_int64): s64 store_entry[c] @ p [[sealed, comment(info)]];
(IndexType::_string): char store_entry[] @ p [[sealed, comment(info)]];
(IndexType::_bin): u8 store_entry[c] @ p [[sealed, comment(info)]];
(IndexType::_string_array): String store_entry[c] @ p [[sealed, comment(info)]];
}
};
struct Header {
HeaderMagic magic;
u8 version;
u8 reserved[4] [[hidden]];
u32 amount [[comment("Amount of Index entries")]];
u32 size [[comment("Header data len")]];
Index index[amount];
auto offset = $;
StoreEntry store[amount];
$ += size;
};
struct RPM {
Lead lead;
std::mem::MagicSearch<"\x8e\xad\xe8", Header> header;
u8 lzma_archive[while(!std::mem::eof())] [[comment("LZMA + CPIO")]];
};
RPM package @ 0x00;

87
patterns/sdb.hexpat Normal file
View File

@@ -0,0 +1,87 @@
#pragma description SDB File (Shim DataBase file)
#pragma author learn_more
#pragma magic [ 73 64 62 66 ] @ 0x08
#pragma endian little
import std.mem;
import std.io;
import std.string;
import type.base;
enum TagType : u8 {
Null = 0x1,
Byte = 0x2,
Word = 0x3,
DWord = 0x4,
QWord = 0x5,
StringRef = 0x6,
List = 0x7,
String = 0x8,
Binary = 0x9,
};
bitfield TypeId {
id : 12;
type : 4;
};
fn get_tag(TypeId tag) {
return std::format("{:x}{:03x}", tag.type, tag.id);
};
fn get_type(TypeId tag) {
TagType tmp = tag.type;
str name = std::format("{}", tmp);
u32 len = std::string::length(name);
return std::string::substr(name, 9, len-9);
};
struct Tag {
TypeId tag[[format_read("get_tag")]];
match (tag.type) {
(TagType::Null) : {
// All set
}
(TagType::Byte) : {
u8 value;
u8 pad[[hidden]];
}
(TagType::Word) : {
u16 value;
}
(TagType::DWord) : {
u32 value;
}
(TagType::QWord) : {
u64 value;
}
(TagType::StringRef) : {
u32 str_offset;
char value @ str_offset;
}
(TagType::List) : {
u32 size ;
Tag tags[while($ < ((addressof(size) + size + 4) ))];
}
(TagType::String) : {
u32 size;
char16 value[size/2];
}
(TagType::Binary) : {
u32 size;
u8 value[size];
if (size & 1 ) {
u8 pad[[hidden]];
}
}
}
}[[name(get_tag(tag)), comment(get_type(tag))]];
struct Header {
u32 Major;
u32 Minor;
char Magic[4];
Tag tags[while(!std::mem::eof())];
};
Header file @ 0x0;

218
patterns/smk.hexpat Normal file
View File

@@ -0,0 +1,218 @@
#pragma author xZise
#pragma description Smacker video file
#pragma endian little
#pragma magic [53 4D 4B 32] @ 0x0
import std.core;
import std.io;
import std.mem;
bitfield HeaderFlags {
ContainsRingFrame : 1;
YInterlaced : 1;
YDoubled : 1;
padding : 29;
};
bitfield HeaderAudioFlags {
isCompressed : 1;
hasAudio : 1;
is16BitAudio : 1;
isStereo : 1;
soundDecompression : 2;
padding : 2;
audioSampleRate : 24;
};
fn format_frame_rate(auto frame_rate) {
float fps;
if (frame_rate > 0) {
fps = 1000.0 / frame_rate;
} else if (frame_rate < 0) {
fps = 100000.0 / (-frame_rate);
} else {
fps = 10;
}
return std::format("{} fps", fps);
};
struct Header {
char Signature[4];
u32 Width;
u32 Height;
u32 FrameCount;
s32 FrameRate [[format("format_frame_rate")]];
HeaderFlags Flags;
u32 AudioSize[7];
u32 TreesSize;
u32 MMap_Size;
u32 MClr_Size;
u32 Full_Size;
u32 Type_Size;
HeaderAudioFlags AudioRate[7];
padding[4];
};
fn format_size(auto size) {
if (size.keyframe != 0) {
return std::format("[K] {} B", size.size);
}
return std::format("[ ] {} B", size.size);
};
bitfield Size {
keyframe : 1;
padding : 1;
dwordCount : 30;
u32 size = dwordCount * 4;
} [[format("format_size")]];
bitfield FrameType {
ContainsPaletteRecord : 1;
ContainsAudioTrack0 : 1;
ContainsAudioTrack1 : 1;
ContainsAudioTrack2 : 1;
ContainsAudioTrack3 : 1;
ContainsAudioTrack4 : 1;
ContainsAudioTrack5 : 1;
ContainsAudioTrack6 : 1;
};
fn format_palette_copy_only(auto copy_only) {
return std::format("copy: {0}", copy_only.copy);
};
bitfield PaletteCopyOnly {
copy: 7;
mode: 1;
} [[format("format_palette_copy_only")]];
fn format_palette_copy_skip(auto copy_skip) {
return std::format("copy: {0}, skip: {1}", copy_skip.copy, copy_skip.skip);
};
bitfield PaletteCopySkip {
copy: 6;
mode: 2;
skip: 8;
} [[format("format_palette_copy_skip")]];
bitfield PaletteColor {
blue: 6 [[color("0000FF")]];
mode: 2;
green: 6 [[color("00FF00")]];
padding: 2;
red: 6 [[color("FF0000")]];
padding: 2;
u8 r8 = red << 2 | red >> 4;
u8 g8 = green << 2 | green >> 4;
u8 b8 = blue << 2 | blue >> 4;
} [[hex::inline_visualize("color", r8, g8, b8, 0xff)]];
fn format_palette_chunk_block(auto entry) {
u8 first = entry.type;
if (first & 0x80 == 0x80) {
return format_palette_copy_only(entry.copy);
} else if (first & 0x40 == 0x40) {
return format_palette_copy_skip(entry.copy_skip);
} else {
return std::format("color: 0x{0:02x}{1:02x}{2:02x}", entry.color.r8, entry.color.g8, entry.color.b8);
}
};
enum PaletteChunkBlockType: u8 {
Color = 0x00 ... 0x3F,
CopySkip = 0x40 ... 0x7F,
CopyOnly = 0x80 ... 0xFF
};
// Unfortunately the match expression does not support ranges, so this is a helper for the following code:
// PaletteChunkBlockType type = std::mem::read_unsigned($, 1)
fn get_palette_chunk_block_type() {
u8 type = std::mem::read_unsigned($, 1);
if (type & 0x80 == 0x80) {
return PaletteChunkBlockType::CopyOnly;
} else if (type & 0x40 == 0x40) {
return PaletteChunkBlockType::CopySkip;
} else {
return PaletteChunkBlockType::Color;
}
};
fn get_last_position() {
PaletteChunkBlockType type = get_palette_chunk_block_type();
match (type) {
(PaletteChunkBlockType::CopyOnly): return $;
(PaletteChunkBlockType::CopySkip): return $ + 1;
(PaletteChunkBlockType::Color): return $ + 2;
}
};
struct PaletteChunkBlock {
PaletteChunkBlockType type = get_palette_chunk_block_type() [[export]];
match (type) {
(PaletteChunkBlockType::CopyOnly): PaletteCopyOnly copy;
(PaletteChunkBlockType::CopySkip): PaletteCopySkip copy_skip;
(PaletteChunkBlockType::Color): PaletteColor color;
}
} [[format("format_palette_chunk_block")]];
struct PaletteChunk {
u8 length;
u128 end = $ + length * 4 - 1;
PaletteChunkBlock blocks[while(get_last_position() < end)];
if ($ < end) {
padding[end - $];
}
};
struct AudioTrack<auto trackIndex> {
u32 length;
if (parent.parent.header.AudioRate[trackIndex].isCompressed) {
u32 decompressedSize;
}
u8 trackData[length - ($ - addressof(this))];
};
struct FramesData {
u32 frame_index = std::core::array_index();
u32 size = parent.sizes[frame_index].size;
if (parent.frameTypes[frame_index].ContainsPaletteRecord) {
PaletteChunk palette;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack0) {
AudioTrack<0> audioTrack0;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack1) {
AudioTrack<1> audioTrack1;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack2) {
AudioTrack<2> audioTrack2;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack3) {
AudioTrack<3> audioTrack3;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack4) {
AudioTrack<4> audioTrack4;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack5) {
AudioTrack<5> audioTrack5;
}
if (parent.frameTypes[frame_index].ContainsAudioTrack6) {
AudioTrack<6> audioTrack6;
}
u8 video[size - ($ - addressof(this))];
};
struct SMK {
Header header;
Size sizes[header.FrameCount];
FrameType frameTypes[header.FrameCount] ;
u8 trees[header.TreesSize];
FramesData frames[header.FrameCount];
};
SMK smk @ 0x00 [[inline]];

935
patterns/stdfv4.hexpat Normal file
View File

@@ -0,0 +1,935 @@
#pragma author andreasWallner
#pragma description STDFv4 (Standard Test Data Format)
#pragma pattern_limit 5000000
// Specification used: http://www.kanwoda.com/wp-content/uploads/2015/05/std-spec.pdf
// Example file used for testing: https://roos.com/docs/RBEH-AH4R8T?Open
import std.mem;
import std.time;
import std.sys;
import type.byte;
import std.io;
namespace format {
fn epoch(auto epoch) {
return std::format("{} ({})", std::time::format(std::time::to_local(epoch)), epoch);
};
fn absent_255(u8 x) {
if (x == 255) {
return "- absent - (255)";
} else {
return x;
}
};
fn absent_neg_32768(s16 x) {
if (x == -32768) {
return "- absent - (-32768)";
} else {
return x;
}
};
fn absent_65_535(u16 x) {
if (x == 65535) {
return "- absent - (65535)";
} else {
return x;
}
};
fn absent_4_294_967_295(u32 x) {
if (x == 4294967295) {
return "- absent - (4,294,967,295)";
} else {
return x;
}
};
fn absent_space(auto x) {
if (x == ' ') {
return "- absent - ( )";
} else {
return x;
}
};
fn absent_0(auto x) {
if (x == 0) {
return "- absent - (0)";
} else {
return x;
}
};
fn absent_1(auto x) {
if (x == 1) {
return "- absent - (1)";
} else {
return x;
}
};
}
fn format_VarLenStr(ref auto vls) {
return std::format("\"{}\"", vls.string);
};
/// Variable length string (C*n in spec)
struct VarLenStr {
u8 len;
char string[len];
} [[format("format_VarLenStr")]];
fn format_VarLenBytes(ref auto vlb) {
return vlb.bytes;
};
/// Variable length bytes string (D*n in spec)
struct VarLenBytes {
u8 len;
u8 bytes[len];
} [[format("format_VarLenBytes")]];
fn format_VarLenBits(ref auto vlb) {
return std::format("{} valid bits", vlb.bitlen);
};
/// Variable length bit string (B*n in spec)
struct VarLenBits {
u16 bitlen;
u8 bits[bitlen/8];
};
fn format_VarLenNibbles(ref auto vln) {
return vlb.n;
};
/// Variable length nibble array (kxN*1 in spec)
struct VarLenNibbles {
u8 len;
type::Nibbles n[(len+1)/2];
};
// See STDF V4, File Attributes Record, p.16
enum CpuType : u8 {
PDP11_VAX = 0,
SUN1234 = 1,
IBM = 2,
};
struct Header {
u16 REC_LEN;
u8 REC_TYPE;
u8 REC_SUB;
} [[static]];
struct UnknownRecord : Header {
u8 data[REC_LEN];
};
// See STDF V4, File Attributes Record, p.16
struct FAR : Header {
CpuType CPU_TYPE [[comment("CPU type that wrote this file")]];
u8 STDF_VER [[comment("STDF version number")]];
} [[static]];
// See STDF V4, Audit Trail Record, p.17
struct ATR : Header {
std::time::EpochTime MOD_TIM [[comment("Date and time of STDF file modification"), format("format::epoch")]];
VarLenStr CMD_LINE [[comment("Command line of program")]];
};
// See STDF V4, Master Information Record, p.18
fn format_MODE_COD(char x) {
match (x) {
('A'): return "AEL (Automatic Edge Lock) mode (A)";
('C'): return "Checker mode (C)";
('D'): return "Development/Debug test mode (D)";
('E'): return "Engineering mode (same as Development mode) (E)";
('M'): return "Maintenance mode (M)";
('P'): return "Production test mode (P)";
('Q'): return "Quality Control (Q)";
(' '): return "- absent -";
(_): return std::format("Unknown station mode: {}", x);
}
};
fn format_RTST_COD(char x) {
match (x) {
('Y'): return "Lot was previously tested (Y)";
('N'): return "Lot has not been previously tested (N)";
(' '): return "Not known if lot has been previously tested";
('0' ... '9'): return std::format("Lot has been tested previously {} times", x);
(_): return std::format("Unknown retest condition: {}", x);
}
};
struct MIR : Header {
auto start = $;
std::time::EpochTime SETUP_T [[comment("Date and time of job setup"), format("format::epoch")]];
std::time::EpochTime START_T [[comment("Date and time first part tested"), format("format::epoch")]];
u8 STAT_NUM [[comment("Tester station number")]];
char MODE_COD [[comment("Test mode code (e.g. prod, dev) (absent if 'space')"), format("format_MODE_COD")]];
char RTST_COD [[comment("Lot retest code (absent if 'space')"), format("format_RTST_COD")]];
char PROT_COD [[comment("Data protection code (absent if 'space')"), format("format::absent_space")]];
u16 BURN_TIM [[comment("Burn-in time (in minutes) (absent if 65,535)"), format("format::absent_65_535")]];
char CMOD_COD [[comment("Command mode code (absent if 'space')"), format("format::absent_space")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr LOT_ID [[comment("Lot ID (customer specified)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PART_TYP [[comment("Part Type (or product ID)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr NODE_NAM [[comment("Name of node that generated data")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TSTR_TYP [[comment("Tester type")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr JOB_NAM [[comment("Job name (test program name)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr JOB_REV [[comment("Job (test program) revision number (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SBLOT_ID [[comment("Sublot ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr OPER_NAM [[comment("Operator name or ID (at setup time) (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr EXEC_TYP [[comment("Tester executive software type (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr EXEC_VER [[comment("Tester exec software version number (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TEST_COD [[comment("Test phase or step code (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TST_TEMP [[comment("Test temperature (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr USER_TXT [[comment("Generic user text (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr AUX_FILE [[comment("Name of auxiliary data file (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PKG_TYP [[comment("Package type (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr FAMLY_ID [[comment("Product family ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr DATE_COD [[comment("Date code (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr FACIL_ID [[comment("Test facility ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr FLOOR_ID [[comment("Test floor ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PROC_ID [[comment("Fabrication process ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr OPER_FRQ [[comment("Operation frequency or step (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SPEC_NAM [[comment("Test specification name (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SPEC_VER [[comment("Test specification version number (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr FLOW_ID [[comment("Test flow ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SETUP_ID [[comment("Test setup ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr DSGN_REV [[comment("Device design revision (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr ENG_ID [[comment("Engineering lot ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr ROM_COD [[comment("ROM code ID (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SERL_NUM [[comment("Tester serial number (absent if length byte=0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SUPR_NAM [[comment("Supervisor name or ID (absent if length byte=0)")]];
};
// See STDF V4, Master Results Record, p.21
struct MRR : Header {
auto start = $;
std::time::EpochTime FINISH_T [[comment("Date and time of the last part tested"), format("format::epoch")]];
if (($ - start) == REC_LEN) { return; }
char DISP_COD[1] [[comment("Lot disposition code")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr USR_DESC [[comment("Lot description supplied by user")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr EXC_DESC [[comment("Lot description supplied by exec")]];
};
// See STDF V4, Part Count Record, p.22
fn format_HEAD_NUM(u8 x) {
if (x == 255) {
return std::format("all test sites (255)");
} else {
return x;
}
};
struct PCR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number"), format("format_HEAD_NUM")]];
u8 SITE_NUM [[comment("Test site number")]];
if (($ - start) == REC_LEN) { return; }
u32 PART_CNT [[comment("Number of parts tested (absent i 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 RTST_CNT [[comment("Number of parts retested (absent i 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 ABRT_CNT [[comment("Number of aborts during testing (absent i 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 GOOD_CNT [[comment("Number of good (passed) parts tested (absent i 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 FUNC_CNT [[comment("Number of functional parts tested (absent i 4,294,967,295)"), format("format::absent_4_294_967_295")]];
} [[static]];
// See STDF V4, Hardware Bin Record, p.23
enum HBRHbinPf : char {
PASS = 'P',
FAIL = 'F',
UNKNOWN = ' '
};
struct HBR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number"), format("format_HEAD_NUM")]];
u8 SITE_NUM [[comment("Test site number")]];
u16 HBIN_NUM [[comment("Hardware bin number")]];
u32 HBIN_CNT [[comment("Number of parts in bin")]];
if (($ - start) == REC_LEN) { return; }
HBRHbinPf HBIN_PF [[comment("Pass/fail indication (absent if 'space')")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr HBIN_NAM [[comment("Name of hardware bin (absent if length byte = 0)")]];
};
// See STDF V4, Software Bin Record, p.25
enum SBRSbinPf : char {
PASS = 'P',
FAIL = 'F',
UNKNOWN = ' '
};
struct SBR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number"), format("format_HEAD_NUM")]];
u8 SITE_NUM [[comment("Test site number")]];
u16 SBIN_NUM [[comment("Software bin number")]];
u32 SBIN_CNT [[comment("Number of parts in bin")]];
if (($ - start) == REC_LEN) { return; }
SBRSbinPf SBIN_PF [[comment("Pass/fail indication (absent if 'space')")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SBIN_NAM [[comment("Name of software bin (absent if length byte = 0)")]];
};
// See STDF V4, Pin Map Record, p.27
fn format_PMR_HEAD_NUM(u8 x) {
if (x == 1) {
return "Unidentified (1)";
} else {
return x;
}
};
struct PMR : Header {
auto start = $;
u16 PMR_INDX [[comment("Unique index associated with pin")]];
if (($ - start) == REC_LEN) { return; }
u16 CHAN_TYP [[comment("Channel type (absent if 0)"), format("format::absent_0")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CHAN_NAM [[comment("Channel name (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PHY_NAM [[comment("Physical name of pin (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr LOG_NAM [[comment("Logical name of pin (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
u8 HEAD_NUM [[comment("Head number associated with channel (absent if 1)"), format("format_PMR_HEAD_NUM")]];
if (($ - start) == REC_LEN) { return; }
u8 SITE_NUM [[comment("Site number associated with channel (absent if 1)"), format("format::absent_1")]];
};
// See STDF V4, Pin Group Record, p.29
struct PGR : Header {
auto start = $;
u16 GRP_INDX [[comment("Unique index associated with pin group")]];
VarLenStr GRP_NAM [[comment("Name of pin group (absent if length byte = 0)")]];
u16 INDX_CNT [[comment("Count (k) of PMR indexes")]];
if (($ - start) == REC_LEN) { return; }
u16 PMR_INDX[INDX_CNT] [[comment("Array of indexes for pins in the group (absent if INDX_CNT = 0)")]];
};
// See STDF V4, Pin List Record, p.30
fn format_PLR_GRP_MODE(u16 x) {
match (x) {
(0): return "Unknown";
(10): return "Normal";
(20): return "SCIO (Same Cycle I/O)";
(21): return "SCIO Midband";
(22): return "SCIO Valid";
(23): return "SCIO Window Sustain";
(30): return "Dual drive (two drive bits per cycle)";
(31): return "Dual drive Midband";
(32): return "Dual drive Valid";
(33): return "Dual drive Window Sustain";
(32767 ... 65535): return std::format("Customer specific use ({})", x);
(_): return std::format("RFU ({})", x);
}
};
enum PLRGrpRadx : u8 {
DEFAULT = 0,
BINARY = 2,
OCTAL = 8,
DECIMAL = 10,
HEX = 16,
SYMBOLIC = 20
};
struct PLR : Header {
auto start = $;
u16 GRP_CNT [[comment("Count (k) of pins or pin groups")]];
if (($ - start) == REC_LEN) { return; }
u16 GRP_INDX[GRP_CNT] [[comment("Array of pin or pin group indexes")]];
if (($ - start) == REC_LEN) { return; }
u16 GRP_MODE[GRP_CNT] [[comment("Operating mode of pin group (absent if 0)"), format("format_PLR_GRP_MODE")]];
if (($ - start) == REC_LEN) { return; }
PLRGrpRadx GRP_RADX[GRP_CNT] [[comment("Display radix of pin group (absent if 0)"), format("format::absent_0")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PGM_CHAR[GRP_CNT] [[comment("Program state encoding characters (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr RTN_CHAR[GRP_CNT] [[comment("Return state encoding characters (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PGM_CHAL[GRP_CNT] [[comment("Program state encoding characters (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr RTN_CHAL[GRP_CNT] [[comment("Return state encoding characters (absent if length byte = 0)")]];
};
// See STDF V4, Retest Data Record, p.32
struct RDR : Header {
auto start = $;
u16 NUM_BINS [[comment("Number (k) of bins being retested")]];
if (($ - start) == REC_LEN) { return; }
u16 RTST_BIN[NUM_BINS] [[comment("Array of retest bin numbers (absent if NUM_BINS = 0)")]];
} [[static]];
// See STDF V4, Site Description Record, p.33
struct SDR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number")]];
u8 SITE_GRP [[comment("Site group number")]];
u8 SITE_CNT [[comment("Number (k) of test sites in site group")]];
u8 SITE_NUM[SITE_CNT] [[comment("Array of test site numbers")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr HAND_TYP [[comment("Handler or prober type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr HAND_ID [[comment("Handler or prober ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CARD_TYP [[comment("Probe card type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CARD_ID [[comment("Probe card ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr LOAD_TYP [[comment("Load board type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr LOAD_ID [[comment("Load board ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr DIB_TYP [[comment("DIB board type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr DIB_ID [[comment("DIB board ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CABL_TYP [[comment("Interface cable type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CABL_ID [[comment("Interface cable ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CONT_TYP [[comment("Handler contactor type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr CONT_ID [[comment("Handler contactor ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr LASR_TYP [[comment("Laser type (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr LASR_ID [[comment("Laser ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr EXTR_TYP [[comment("Extra equipment type field (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr EXTR_ID [[comment("Extra equipment ID (absent if length byte = 0)")]];
};
// See STDF V4, Wafer Information Record, p.25
fn format_WIR_SITE_GRP(u8 x) {
if (x == 255) {
return "Unknown or unsupported (255)";
} else {
return x;
}
};
struct WIR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number")]];
u8 SITE_GRP [[comment("Site group number (absent if '255')"), format("format_WIR_SITE_GRP")]];
u32 START_T [[comment("Date and time first part tested")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr WAFER_ID [[comment("Wafer ID (absent if length byte = 0)")]];
};
// See STDF V4, Wafer Results Record, p.36
struct WRR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number")]];
u8 SITE_GRP [[comment("Site group number (absent if '255')"), format("format::absent_255")]];
u32 FINISH_T [[comment("Date and time last part tested")]];
u32 PART_CNT [[comment("Number of parts tested")]];
if (($ - start) == REC_LEN) { return; }
u32 RTST_CNT [[comment("Number of parts retested (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 ABRT_CNT [[comment("Number of aborts during testing (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 GOOD_CNT [[comment("Number of good (passed) parts tested (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 FUNC_CNT [[comment("Number of functional parts tested (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr WAFER_ID [[comment("Wafer ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr FABWF_ID [[comment("Fab wafer ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr FRAME_ID [[comment("Wafer frame ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr MASK_ID [[comment("Wafer mask ID (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr USR_DESC [[comment("Wafer description supplied by user (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr EXC_DESC [[comment("Wafer description supplied by exec (absent if length byte = 0)")]];
};
// See STDF V4, Wafer Configuration Record, p.38
enum WCRWfUnits : u8 {
Unknown = 0,
INCH = 1,
CM = 2,
MM = 3,
MIL = 4,
};
enum WCRWfFlat : char {
UP = 'U',
DOWN = 'D',
LEFT = 'L',
RIGHT = 'R',
UNKNOWN = ' '
};
enum WCRPosX : char {
LEFT = 'L',
RIGHT = 'R',
UNKNOWN = ' '
};
enum WCRPosY : char {
UP = 'U',
DOWN = 'D',
UNKNOWN = ' '
};
struct WCR : Header {
auto start = $;
if (($ - start) == REC_LEN) { return; }
float WAFR_SIZ [[comment("Diameter of wafer in WF_UNITS (absent if 0)"), format("format::absent_0")]];
if (($ - start) == REC_LEN) { return; }
float DIE_HT [[comment("Height of die in WF_UNITS (absent if 0)"), format("format::absent_0")]];
if (($ - start) == REC_LEN) { return; }
float DIE_WID [[comment("Width of die in WF_UNITS (absent if 0)"), format("format::absent_0")]];
if (($ - start) == REC_LEN) { return; }
WCRWfUnits WF_UNITS [[comment("Units for wafer and die dimensions (absent if 0)"), format("format::absent_0")]];
if (($ - start) == REC_LEN) { return; }
WCRWfFlat WF_FLAT [[comment("Orientation of wafer flat (absent if 'space')"), format("format::absent_space")]];
if (($ - start) == REC_LEN) { return; }
s16 CENTER_X [[comment("X coordinate of center die on wafer (invalid if -32768"), format("format::absent_neg_32768")]];
if (($ - start) == REC_LEN) { return; }
s16 CENTER_Y [[comment("Y coordinate of center die on wafer (invalid if -32768"), format("format::absent_neg_32768")]];
if (($ - start) == REC_LEN) { return; }
WCRPosX POS_X [[comment("Positive X direction of wafer (absent if 'space')"), format("format::absent_space")]];
if (($ - start) == REC_LEN) { return; }
WCRPosY POS_Y [[comment("Positive Y direction of wafer (absent if 'space')"), format("format::absent_space")]];
} [[static]];
// See STDF V4, Part Information Record, p.40
fn format_PIR_PRR_x_NUM(u8 x) {
if (x == 1) {
return "No parallel testing or no test site/head identification";
} else {
return x;
}
};
struct PIR : Header {
u8 HEAD_NUM [[comment("Test head number"), format("format_PIR_PRR_x_NUM")]];
u8 SITE_NUM [[comment("Test site number"), format("format_PIR_PRR_x_NUM")]];
} [[static]];
// See STDF V4, Part Results Record, p.41
bitfield PRRPartFlg {
supersedes_prev_same_part_id: 1 [[comment("0 = This is a new part. Its data device does not supersede that of any previous device.\n\n1 = The PIR, PTR, MPR, FTR, and PRR records that make up the current sequence (identified as having the same HEAD_NUM and SITE_NUM) supersede any previous sequence of records with the same PART_ID. (A repeated part sequence usually indicates a mistested part.)")]];
supersedes_prev_same_xy: 1 [[comment("0 = This is a new part. Its data device does not supersede that of any previous device.\n\n1 = The PIR, PTR, MPR, FTR, and PRR records that make up the current sequence (identified as having the same HEAD_NUM and SITE_NUM) supersede any previous sequence of records with the same X_COORD and Y_COORD. (A repeated part sequence usually indicates a mistested part.)")]];
abnormal_test_end: 1 [[comment("0 = Part testing completed normally\n\n1 = Abnormal end of testing")]];
part_failed: 1 [[comment("0 = Part passed\n\n1 = Part failed")]];
no_pass_fail: 1 [[comment("0 = Pass/fail flag (bit 3) is valid\n\n1 = Device completed testing with no pass/fail indication (i.e., bit 3 is invalid)")]];
};
struct PRR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number"), format("format_PIR_PRR_x_NUM")]];
u8 SITE_NUM [[comment("Test site number"), format("format_PIR_PRR_x_NUM")]];
PRRPartFlg PART_FLG [[comment("Part information flag")]];
u16 NUM_TEST [[comment("Number of tests executed")]];
u16 HARD_BIN [[comment("Hardware bin number")]];
if (($ - start) == REC_LEN) { return; }
u16 SOFT_BIN [[comment("Software bin number (absent if 65535)"), format("format::absent_65_535")]];
if (($ - start) == REC_LEN) { return; }
s16 X_COORD [[comment("(Wafer) X coordinate (absent if -32768"), format("format::absent_neg_32768")]];
if (($ - start) == REC_LEN) { return; }
s16 Y_COORD [[comment("(Wafer) Y coordinate (absent if -32768"), format("format::absent_neg_32768")]];
if (($ - start) == REC_LEN) { return; }
u32 TEST_T [[comment("Elapsed test time in milliseconds 0")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PART_ID [[comment("Part identification (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PART_TXT [[comment("Part description text (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
type::Nibbles PART_FIX [[comment("Part repair information (absent if length byte = 0)")]];
};
// See STDF V4, Test Synopsis Record, p.43
fn format_TSR_HEAD_NUM(u8 x) {
if (x == 255) {
return "Summary for all test sites";
} else {
return x;
}
};
enum TSRTestTyp : char {
PARAMETRIC = 'P',
FUNCTIONAL = 'F',
MULTIPLE_RESULT = 'M',
UNKNOWN = ' ',
};
bitfield TSROptFlag {
TEST_MIN_invalid: 1;
TEST_MAX_invalid: 1;
TEST_TIM_invalid: 1;
padding: 1;
TST_SUMS_invalid: 1;
TST_SQRS_invalid: 1;
padding: 2;
};
struct TSR : Header {
auto start = $;
u8 HEAD_NUM [[comment("Test head number See note"), format("format_TSR_HEAD_NUM")]];
u8 SITE_NUM [[comment("Test site number")]];
TSRTestTyp TEST_TYP [[comment("Test type (absent if 'space')"), format("format::absent_space")]];
u32 TEST_NUM [[comment("Test number")]];
if (($ - start) == REC_LEN) { return; }
u32 EXEC_CNT [[comment("Number of test executions (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 FAIL_CNT [[comment("Number of test failures (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
u32 ALRM_CNT [[comment("Number of alarmed tests (absent if 4,294,967,295)"), format("format::absent_4_294_967_295")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TEST_NAM [[comment("Test name (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr SEQ_NAME [[comment("Sequencer (program segment/flow) name (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TEST_LBL [[comment("Test label or text (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
TSROptFlag OPT_FLAG [[comment("Optional data flag")]];
if (($ - start) == REC_LEN) { return; }
float TEST_TIM [[comment("Average test execution time in seconds (invalid if OPT_FLAG bit 2 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float TEST_MIN [[comment("Lowest test result value (invalid if OPT_FLAG bit 0 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float TEST_MAX [[comment("Highest test result value (invalid if OPT_FLAG bit 1 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float TST_SUMS [[comment("Sum of test result values (invalid if OPT_FLAG bit 4 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float TST_SQRS [[comment("Sum of squares of test result values (invalid if OPT_FLAG bit 5 = 1)")]];
};
// See STDF V4, Parametric Test Record, p.45
bitfield PTRTestFlg {
alarm: 1 [[comment("0 = No alarm\n\n1 = Alarm detected during testing")]];
result_valid: 1 [[comment("0 = The value in the RESULT field is valid (see note on RESULT)\n\n1 = The value in the RESULT field is not valid. This setting indicates that the test was executed, but no datalogged value was taken. You should read bits 6 and 7 of TEST_FLG to determine if the test passed or failed.")]];
unreliable: 1 [[comment("0 = Test result is reliable\n\n1 = Test result is unreliable")]];
timeout: 1 [[comment(" 0 = No timeout\n\n1 = Timeout occurred")]];
executed: 1 [[comment("0 = Test was executed\n\n1 = Test not executed")]];
aborted: 1 [[comment("0 = No abort\n\n1 = Test aborted")]];
pass_fail_valid: 1 [[comment("0 = Pass/fail flag (bit 7) is valid\n\n1 = Test completed with no pass/fail indication")]];
failed: 1 [[comment("0 = Test passed\n\n1 = Test failed")]];
} [[static]];
bitfield ParamFlg {
scale_error: 1 [[comment("0 = No scale error\n\n1 = Scale error")]];
drift_error: 1 [[comment("0 = No drift error\n\n1 = Drift error (unstable measurement)")]];
oscillation_detected: 1 [[comment("0 = No oscillation\n\n1 = Oscillation detected")]];
gt_test_limit: 1 [[comment("0 = Measured value not high\n\n1 = Measured value higher than high test limit")]];
lt_test_limit: 1 [[comment("0 = Measured value not low\n\n1 = Measured value lower than low test limit")]];
passed_alternate_limits: 1 [[comment("0 = Test failed or test passed standard limits\n\n1 = Test passed alternate limits")]];
eq_low_is_pass: 1 [[comment("0 = If result = low limit, then result is “fail.”\n\n1 = If result = low limit, then result is “pass.”")]];
eq_high_is_pass: 1 [[comment("0 = If result = high limit, then result is “fail.”\n\n1 = If result = high limit, then result is “pass.”")]];
} [[static]];
bitfield PTROptFlag {
res_scal_invalid: 1 [[comment("set = RES_SCAL value is invalid. The default set by the first PTR with this test number will be used.")]];
padding: 1;
no_lo_spec_limit: 1 [[comment("set = No low specification limit.")]];
no_hi_spec_limit: 1 [[comment("set = No high specification limit.")]];
lo_limit_llm_scal_invalid: 1 [[comment("set = LO_LIMIT and LLM_SCAL are invalid. The default values set for these fields in the first PTR with this test number will be used.")]];
hi_limit_hlm_scal_invalid: 1 [[comment("set = HI_LIMIT and HLM_SCAL are invalid. The default values set for these fields in the first PTR with this test number will be used.")]];
no_low_limit: 1 [[comment("No Low Limit for this test (LO_LIMIT and LLM_SCAL are invalid).")]];
no_high_limit: 1 [[comment("No High Limit for this test (HI_LIMIT and HLM_SCAL are invalid).")]];
} [[static]];
struct PTR : Header {
u64 start = $;
u32 TEST_NUM [[comment("Test number")]];
u8 HEAD_NUM [[comment("Test head number")]];
u8 SITE_NUM [[comment("Test site number")]];
PTRTestFlg TEST_FLG [[comment("Test flags (fail, alarm, etc.)")]];
ParamFlg PARM_FLG [[comment("Parametric test flags (drift, etc.)")]];
if (($ - start) == REC_LEN) { return; }
float RESULT [[comment("Test result (only valid if TEST_FLG bit 1 = 1)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TEST_TXT [[comment("Test description text or label (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr ALARM_ID [[comment("Name of alarm (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
PTROptFlag OPT_FLAG [[comment("Optional data flag See note")]];
if (($ - start) == REC_LEN) { return; }
s8 RES_SCAL [[comment("Test results scaling exponent (invalid if OPT_FLAG bit 0 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s8 LLM_SCAL [[comment("Low limit scaling exponent (invalid if OPT_FLAG bit 4 or 6 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s8 HLM_SCAL [[comment("High limit scaling exponent (invalid if OPT_FLAG bit 5 or 7 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float LO_LIMIT [[comment("Low test limit value (invalid if OPT_FLAG bit 4 or 6 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float HI_LIMIT [[comment("High test limit value (invalid if OPT_FLAG bit 5 or 7 = 1)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr UNITS [[comment("Test units (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr C_RESFMT [[comment("ANSI C result format string (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr C_LLMFMT [[comment("ANSI C low limit format string (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr C_HLMFMT [[comment("ANSI C high limit format string (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
float LO_SPEC [[comment("Low specification limit value (invalid if OPT_FLAG bit 2 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float HI_SPEC [[comment("High specification limit value (invalid if OPT_FLAG bit 3 = 1)")]];
};
// See STDF V4, Multiple-Result Parametric Record, p.51
bitfield MPR_FTR_TestFlg {
alarm: 1 [[comment("0 = No alarm\n\n1 = Alarm detected during testing")]];
padding: 1;
results_unreliable: 1 [[comment("0 = Test results are reliable\n\n1 = Test results are unreliable")]];
timeout: 1 [[comment("0 = No timeout\n\n1 = Timeout occurred")]];
test_not_executed: 1 [[comment("0 = Test was executed\n\n1 = Test not executed")]];
test_aborted: 1 [[comment("0 = No abort\n\n1 = Test aborted")]];
completed_no_pass_fail: 1 [[comment("0 = Pass/fail flag (bit 7) is valid\n\n1 = Test completed with no pass/fail indication")]];
test_failed: 1 [[comment("0 = Test passed\n\n1 = Test failed")]];
} [[static]];
bitfield MPROptFlag {
res_scal_invalid: 1 [[comment("set = RES_SCAL value is invalid. The default set by the first MPR with this test number will be used.")]];
start_incr_in_incr_invalid: 1 [[comment("set = START_IN and INCR_IN are invalid.")]];
no_lo_spec_limit: 1 [[comment("set = No low specification limit.")]];
no_hi_spec_limit: 1 [[comment("set = No high specification limit.")]];
lo_limit_llm_scal_invalid: 1 [[comment("set = LO_LIMIT and LLM_SCAL are invalid. The default values set for these fields in the first MPR with this test number will be used.")]];
hi_limit_hlm_scal_invalid: 1 [[comment("set = HI_LIMIT and HLM_SCAL are invalid. The default values set for these fields in the first MPR with this test number will be used.")]];
no_lo_limit: 1 [[comment("set = No Low Limit for this test (LO_LIMIT and LLM_SCAL are invalid).")]];
no_hi_limit: 1 [[comment("set = No High Limit for this test (HI_LIMIT and HLM_SCAL are invalid).")]];
} [[static]];
struct MPR : Header {
auto start = $;
u32 TEST_NUM [[comment("Test number")]];
u8 HEAD_NUM [[comment("Test head number")]];
u8 SITE_NUM [[comment("Test site number")]];
MPR_FTR_TestFlg TEST_FLG [[comment("Test flags (fail, alarm, etc.)")]];
ParamFlg PARM_FLG [[comment("Parametric test flags (drift, etc.)")]];
u16 RTN_ICNT [[comment("Count (j) of PMR indexes")]];
u16 RSLT_CNT [[comment("Count (k) of returned results")]];
if (($ - start) == REC_LEN) { return; }
type::Nibbles RTN_STAT[(RTN_ICNT+1)/2] [[comment("Array of returned states (absent if RTN_ICNT = 0)")]];
if (($ - start) == REC_LEN) { return; }
float RTN_RSLT[RSLT_CNT] [[comment("Array of returned results (absent if RSLT_CNT = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TEST_TXT [[comment("Descriptive text or label (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr ALARM_ID [[comment("Name of alarm (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
MPROptFlag OPT_FLAG [[comment("Optional data flag")]];
if (($ - start) == REC_LEN) { return; }
s8 RES_SCAL [[comment("Test result scaling exponent (absent if OPT_FLAG bit 0 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s8 LLM_SCAL [[comment("Test low limit scaling exponent (absent if OPT_FLAG bit 4 or 6 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s8 HLM_SCAL [[comment("Test high limit scaling exponent (absent if OPT_FLAG bit 5 or 7 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float LO_LIMIT [[comment("Test low limit value (absent if OPT_FLAG bit 4 or 6 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float HI_LIMIT [[comment("Test high limit value (absent if OPT_FLAG bit 5 or 7 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float START_IN [[comment("Starting input value (condition) (absent if OPT_FLAG bit 1 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float INCR_IN [[comment("Increment of input condition (absent if OPT_FLAG bit 1 = 1)")]];
if (($ - start) == REC_LEN) { return; }
u16 RTN_INDX[RTN_ICNT] [[comment("Array of PMR indexes")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr UNITS [[comment("Units of returned results (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr UNITS_IN [[comment("Input condition units (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr C_RESFMT [[comment("ANSI C result format string (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr C_LLMFMT [[comment("ANSI C low limit format string (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr C_HLMFMT [[comment("ANSI C high limit format string (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
float LO_SPEC [[comment("Low specification limit value (absent if OPT_FLAG bit 2 = 1)")]];
if (($ - start) == REC_LEN) { return; }
float HI_SPEC [[comment("High specification limit value (absent if OPT_FLAG bit 3 = 1)")]];
};
// See STDF V4, Functional Test Record, p.55
bitfield FTROptFlag {
CYCL_CNT_invalid: 1 [[comment("set = CYCL_CNT data is invalid")]];
REL_VADR_invalid: 1 [[comment("set = REL_VADR data is invalid")]];
REPT_CNT_invalid: 1 [[comment("set = REPT_CNT data is invalid")]];
NUM_FAIL_invalid: 1 [[comment("set = NUM_FAIL data is invalid")]];
XFAIL_AD_YFAIL_AD_invalid: 1 [[comment("set = XFAIL_AD and YFAIL_AD data are invalid")]];
VECT_OFF_invalid: 1 [[comment("set = VECT_OFF data is invalid (offset defaults to 0)")]];
padding: 2;
} [[static]];
struct FTR : Header {
auto start = $;
u32 TEST_NUM [[comment("Test number")]];
u8 HEAD_NUM [[comment("Test head number")]];
u8 SITE_NUM [[comment("Test site number")]];
MPR_FTR_TestFlg TEST_FLG [[comment("Test flags (fail, alarm, etc.)")]];
if (($ - start) == REC_LEN) { return; }
FTROptFlag OPT_FLAG [[comment("Optional data flag")]];
if (($ - start) == REC_LEN) { return; }
u32 CYCL_CNT [[comment("Cycle count of vector (absent if OPT_FLAG bit 0 = 1)")]];
if (($ - start) == REC_LEN) { return; }
u32 REL_VADR [[comment("Relative vector address (absent if OPT_FLAG bit 1 = 1)")]];
if (($ - start) == REC_LEN) { return; }
u32 REPT_CNT [[comment("Repeat count of vector (absent if OPT_FLAG bit 2 = 1)")]];
if (($ - start) == REC_LEN) { return; }
u32 NUM_FAIL [[comment("Number of pins with 1 or more failures (absent if OPT_FLAG bit 3 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s32 XFAIL_AD [[comment("X logical device failure address (absent if OPT_FLAG bit 4 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s32 YFAIL_AD [[comment("Y logical device failure address (absent if OPT_FLAG bit 4 = 1)")]];
if (($ - start) == REC_LEN) { return; }
s16 VECT_OFF [[comment("Offset from vector of interest (absent if OPT_FLAG bit 5 = 1)")]];
if (($ - start) == REC_LEN) { return; }
u16 RTN_ICNT [[comment("Count (j) of return data PMR indexes")]];
if (($ - start) == REC_LEN) { return; }
u16 PGM_ICNT [[comment("Count (k) of programmed state indexes")]];
if (($ - start) == REC_LEN) { return; }
u16 RTN_INDX[RTN_ICNT] [[comment("Array of return data PMR indexes")]];
if (($ - start) == REC_LEN) { return; }
u8 RTN_STAT[(RTN_ICNT+1)/2] [[comment("Array of returned states\n\n0 = 0 or low\n1 = 1 or high\n2 = midband\n3 = glitch\n4 = undetermined\n5 = failed low\n6 = failed high\n7 = failed midband\n8 = failed with a glitch\n9 = open\nA = short")]];
if (($ - start) == REC_LEN) { return; }
u16 PGM_INDX[PGM_ICNT] [[comment("Array of programmed state indexes")]];
if (($ - start) == REC_LEN) { return; }
u8 PGM_STAT[(PGM_ICNT+1)/2] [[comment("Array of programmed states")]];
if (($ - start) == REC_LEN) { return; }
VarLenNibbles FAIL_PIN [[comment("Failing pin bitfield (absent if length bytes = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr VECT_NAM [[comment("Vector module pattern name (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TIME_SET [[comment("Time set name (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr OP_CODE [[comment("Vector Op Code (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr TEST_TXT [[comment("Descriptive text or label (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr ALARM_ID [[comment("Name of alarm (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr PROG_TXT [[comment("Additional programmed information (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
VarLenStr RSLT_TXT [[comment("Additional result information (absent if length byte = 0)")]];
if (($ - start) == REC_LEN) { return; }
u8 PATG_NUM [[comment("Pattern generator number (absent if 255)"), format("format::absent_255")]];
if (($ - start) == REC_LEN) { return; }
VarLenNibbles SPIN_MAP [[comment("Bit map of enabled comparators (absent if length byte = 0)")]];
};
// See STDF V4, Begin Program Record, p.60
struct BPS : Header {
VarLenStr SEQ_NAME [[comment("Program section (or sequencer) name (absent if length byte = 0)")]];
};
// See STDF V4, End Program Section Record, p.61
struct EPS : Header {} [[static]];
// See STDF V4, Generic Data Record, p.62
enum GenDataType : u8 {
PAD = 0,
U1 = 1,
U2 = 2,
U4 = 3,
I1 = 4,
I2 = 5,
I4 = 6,
R4 = 7,
R8 = 8,
Cn = 10,
Bn = 11,
Dn = 12,
N1 = 13,
} [[static]];
struct GenData {
u8 type;
match (type) {
(GenDataType::PAD): return;
(GenDataType::U1): u8 data;
(GenDataType::U2): u16 data;
(GenDataType::U4): u32 data;
(GenDataType::I1): s8 data;
(GenDataType::I2): s16 data;
(GenDataType::I4): s32 data;
(GenDataType::R4): float data;
(GenDataType::R8): double data;
(GenDataType::Cn): VarLenStr data;
(GenDataType::Bn): VarLenBytes data;
(GenDataType::Dn): VarLenBits data;
(GenDataType::N1): type::Nibbles data;
}
};
struct GDR : Header {
auto start = $;
u16 FLD_CNT [[comment("Count of data fields in record")]];
if (($ - start) == REC_LEN) { return; }
VarLenBytes GEN_DATA [[comment("Data type code and data for one field")]];
};
// See STDF V4, Datalog Text Record, p.64
struct DTR : Header {
VarLenStr TEXT_DAT [[comment("ASCII text string")]];
};
struct Record {
u64 start = $;
Header hh [[hidden, no_unique_address]];
match (hh.REC_TYPE, hh.REC_SUB) {
(0, 10): FAR FAR [[comment("Contains the information necessary to determine how to decode the STDF data contained in the file.")]];
(0, 20): ATR ATR [[comment("Used to record any operation that alters the contents of the STDF file. The name of the program and all its parameters should be recorded in the ASCII field provided in this record. Typically, this record will be used to track filter programs that have been applied to the data.")]];
(1, 10): MIR MIR [[comment("The MIR and the MRR (Master Results Record) contain all the global information that is to be stored for a tested lot of parts. Each data stream must have exactly one MIR, immediately after the FAR (and the ATRs, if they are used). This will allow any data reporting or analysis programs access to this information in the shortest possible amount of time.")]];
(1, 20): MRR MRR [[comment("The Master Results Record (MRR) is a logical extension of the Master Information Record (MIR). The data can be thought of as belonging with the MIR, but it is not available when the tester writes the MIR information. Each data stream must have exactly one MRR as the last record in the data stream.")]];
(1, 30): PCR PCR [[comment("Contains the part count totals for one or all test sites. Each data stream must have at least one PCR to show the part count.")]];
(1, 40): HBR HBR [[comment("Stores a count of the parts “physically” placed in a particular bin after testing. (In wafer testing, “physical” binning is not an actual transfer of the chip, but rather is represented by a drop of ink or an entry in a wafer map file.) This bin count can be for a single test site (when parallel testing) or a total for all test sites. The STDF specification also supports a Software Bin Record (SBR) for logical binning categories. A part is “physically” placed in a hardware bin after testing. A part can be “logically” associated with a software bin during or after testing.")]];
(1, 50): SBR SBR [[comment("Stores a count of the parts associated with a particular logical bin after testing. This bin count can be for a single test site (when parallel testing) or a total for all test sites. The STDF specification also supports a Hardware Bin Record (HBR) for actual physical binning. A part is “physically” placed in a hardware bin after testing. A part can be “logically” associated with a software bin during or after testing.")]];
(1, 60): PMR PMR [[comment("Provides indexing of tester channel names, and maps them to physical and logical pin names. Each PMR defines the information for a single channel/pin combination.")]];
(1, 62): PGR PGR [[comment("Associates a name with a group of pins.")]];
(1, 63): PLR PLR [[comment("Defines the current display radix and operating mode for a pin or pin group.")]];
(1, 70): RDR RDR [[comment("Signals that the data in this STDF file is for retested parts. The data in this record, combined with information in the MIR, tells data filtering programs what data to replace when processing retest data.")]];
(1, 80): SDR SDR [[comment("Contains the configuration information for one or more test sites, connected to one test head, that compose a site group.")]];
(2, 10): WIR WIR [[comment("Acts mainly as a marker to indicate where testing of a particular wafer begins for each wafer tested by the job plan. The WIR and the Wafer Results Record (WRR) bracket all the stored information pertaining to one tested wafer. This record is used only when testing at wafer probe. A WIR/WRR pair will have the same HEAD_NUM and SITE_GRP values.")]];
(2, 20): WRR WRR [[comment("Contains the result information relating to each wafer tested by the job plan. The WRR and the Wafer Information Record (WIR) bracket all the stored information pertaining to one tested wafer. This record is used only when testing at wafer probe time. A WIR/WRR pair will have the same HEAD_NUM and SITE_GRP values.")]];
(2, 30): WCR WCR [[comment("Contains the configuration information for the wafers tested by the job plan. The WCR provides the dimensions and orientation information for all wafers and dice in the lot. This record is used only when testing at wafer probe time.")]];
(5, 10): PIR PIR [[comment("Acts as a marker to indicate where testing of a particular part begins for each part tested by the test program. The PIR and the Part Results Record (PRR) bracket all the stored information pertaining to one tested part.")]];
(5, 20): PRR PRR [[comment("Contains the result information relating to each part tested by the test program. The PRR and the Part Information Record (PIR) bracket all the stored information pertaining to one tested part.")]];
(10, 30): TSR TSR [[comment("Contains the test execution and failure counts for one parametric or functional test in the test program. Also contains static information, such as test name. The TSR is related to the Functional Test Record (FTR), the Parametric Test Record (PTR), and the Multiple Parametric Test Record (MPR) by test number, head number, and site number.")]];
(15, 10): PTR PTR [[comment("Contains the results of a single execution of a parametric test in the test program. The first occurrence of this record also establishes the default values for all semi-static information about the test, such as limits, units, and scaling. The PTR is related to the Test Synopsis Record (TSR) by test number, head number, and site number.")]];
(15, 15): MPR MPR [[comment("Contains the results of a single execution of a parametric test in the test program where that test returns multiple values. The first occurrence of this record also establishes the default values for all semi-static information about the test, such as limits, units, and scaling. The MPR is related to the Test Synopsis Record (TSR) by test number, head number, and site number.")]];
(15, 20): FTR FTR [[comment("Contains the results of the single execution of a functional test in the test program. The first occurrence of this record also establishes the default values for all semi-static information about the test. The FTR is related to the Test Synopsis Record (TSR) by test number, head number, and site number.")]];
(20, 10): BPS BPS [[comment("Marks the beginning of a new program section (or sequencer) in the job plan.")]];
(20, 20): EPS EPS [[comment("Marks the end of the current program section (or sequencer) in the job plan.")]];
(50, 10): GDR GDR [[comment("Contains information that does not conform to any other record type defined by the STDF specification. Such records are intended to be written under the control of job plans executing on the tester. This data may be used for any purpose that the user desires.")]];
(50, 30): DTR DTR [[comment("Contains text information that is to be included in the datalog printout. DTRs may be written under the control of a job plan: for example, to highlight unexpected test results. They may also be generated by the tester executive software: for example, to indicate that the datalog sampling rate has changed. DTRs are placed as comments in the datalog listing.")]];
(_, _): UnknownRecord;
}
std::assert($ == (start + hh.REC_LEN + 4), std::format("Inconsistent record size at offset {}", start));
};
struct STDF {
le FAR far_try [[hidden, no_unique_address]];
if (far_try.REC_LEN == 0x0002 && far_try.REC_TYPE == 0 && far_try.REC_SUB == 10 && far_try.CPU_TYPE == CpuType::IBM) {
std::core::set_endian(std::mem::Endian::Little);
} else if(far_try.REC_LEN == 0x0200 && far_try.REC_TYPE == 0 && far_try.REC_SUB == 10 && far_try.CPU_TYPE == CpuType::SUN1234) {
std::core::set_endian(std::mem::Endian::Big);
} else if(far_try.REC_LEN == 0x01200 && far_try.REC_TYPE == 0 && far_try.REC_SUB == 10 && far_try.CPU_TYPE == CpuType::PDP11_VAX) {
std::error("Mixed endianess of PDP11 / VAX not supported");
} else {
std::warning("No valid FAR found at beginning of the file, endiness unknown - defaulting to native endianess");
}
std::assert(far_try.STDF_VER == 4, "Pattern is for STDF version 4, file indicates a different version in FAR");
Record records[while(!std::mem::eof())];
};
STDF stdf @ 0x00;

50
patterns/tarc.hexpat Normal file
View File

@@ -0,0 +1,50 @@
#pragma author Lexi Mayfield
#pragma description KEX Engine TARC format
#pragma magic [0x54 0x41 0x52 0x43] @ 0x00
enum PixelFormat : u16 {
DXT1 = 14,
DXT5 = 16,
BC7 = 33,
};
struct TARCEntry1 {
char fileName[64];
u64 offset;
u32 size;
u16 width;
u16 height;
u16 numMipMaps;
PixelFormat pixelFormat;
};
struct TARCEntry2 {
char fileName[64];
u64 offset;
u32 size;
u16 x;
u16 y;
u16 width;
u16 height;
u16 numMipMaps;
PixelFormat pixelFormat;
u16 origWidth;
u16 origHeight;
s16 offsetX;
s16 offsetY;
};
struct TARC {
char id[4];
u32 version;
u32 numEntries;
u64 pixelDataOffset;
if (version == 1) {
TARCEntry1 entries[numEntries];
} else if (version == 2) {
TARCEntry2 entries[numEntries];
}
};
TARC tarc @ 0x00;

149
patterns/thumbcache.hexpat Normal file
View File

@@ -0,0 +1,149 @@
#pragma author MrMcX
#pragma description Windows thumbcache files
/*
Thanks to Joachim Metz for his work that helped improve the pattern:
https://github.com/libyal/libwtcdb/blob/main/documentation/Windows%20Explorer%20Thumbnail%20Cache%20database%20format.asciidoc
*/
#pragma magic [0x43 0x4D 0x4D 0x4D] @ 0x00
#pragma endian little
import std.mem;
import type.magic;
import hex.core;
import std.string;
enum WinVer : u32 {
WIN_VISTA = 20,
WIN_7 = 21,
WIN_8 = 30,
WIN_8_1 = 31,
WIN_10 = 32
};
enum Win7Vista_Type : u32 {
THUMBCACHE_32_DB = 0x0,
THUMBCACHE_96_DB = 0x1,
THUMBCACHE_256_DB = 0x2,
THUMBCACHE_1024_DB = 0x3,
THUMBCACHE_SR_DB = 0x4
};
enum Win8_Type: u32 {
THUMBCACHE_16_DB = 0x0,
THUMBCACHE_32_DB = 0x1,
THUMBCACHE_48_DB = 0x2,
THUMBCACHE_96_DB = 0x3,
THUMBCACHE_256_DB = 0x4,
THUMBCACHE_1024_DB = 0x5,
THUMBCACHE_SR_DB = 0x6,
THUMBCACHE_WIDE_DB = 0x7,
THUMBCACHE_EXIF_DB = 0x8
};
enum Win81_Type: u32 {
THUMBCACHE_16_DB = 0x0,
THUMBCACHE_32_DB = 0x1,
THUMBCACHE_48_DB = 0x2,
THUMBCACHE_96_DB = 0x3,
THUMBCACHE_256_DB = 0x4,
THUMBCACHE_1024_DB = 0x5,
THUMBCACHE_1600_DB = 0x6,
THUMBCACHE_SR_DB = 0x7,
THUMBCACHE_WIDE_DB = 0x8,
THUMBCACHE_EXIF_DB = 0x9,
THUMBCACHE_WIDE_ALTERNATE_DB = 0xA
};
enum Win10_Type: u32 {
THUMBCACHE_16_DB = 0x0,
THUMBCACHE_32_DB = 0x1,
THUMBCACHE_48_DB = 0x2,
THUMBCACHE_96_DB = 0x3,
THUMBCACHE_256_DB = 0x4,
THUMBCACHE_768_DB = 0x5,
THUMBCACHE_1280_DB = 0x6,
THUMBCACHE_1920_DB = 0x7,
THUMBCACHE_2560_DB = 0x8,
THUMBCACHE_WIDE_DB = 0x9,
THUMBCACHE_SR_DB = 0xA,
THUMBCACHE_EXIF_DB = 0xB,
THUMBCACHE_WIDE_ALTERNATE_DB = 0xC,
THUMBCACHE_CUSTOM_STREAM_DB = 0xD,
};
struct ThumbCacheHeader {
type::Magic<"CMMM"> signature;
WinVer version;
if(version == WinVer::WIN_10) {
Win10_Type type;
} else if(version == WinVer::WIN_8_1) {
Win81_Type type;
} else if(version == WinVer::WIN_8) {
Win8_Type type;
} else if(version <= WinVer::WIN_7) {
Win7Vista_Type type;
} else {
u32 type;
std::print("Unknown cache type: {}", type);
}
if(version >= WinVer::WIN_8) {
u32;
}
u32 header_size;
u32 offset_to_first_empty;
if(version <= WinVer::WIN_7) {
u32 count;
}
};
struct CacheRecord {
char signature[4];
if (signature != "CMMM") {
std::warning(std::format("Invalid cache entry at 0x{:08x}, skipping: {} ", $, type::escape_bytes(signature)));
padding[while(std::mem::read_string($, 4) != "CMMM")];
continue;
}
u32 size;
u64 entry_hash;
if(header.version == WinVer::WIN_VISTA) {
char16 extension[4];
}
u32 name_size;
u32 padding_size;
u32 data_size;
if(header.version == WinVer::WIN_10) {
u32 width;
u32 height;
}
u32;
u64 data_checksum;
u64 header_checksum;
char16 name[(name_size / 2)];
if(padding_size > 0) {
padding[padding_size];
}
if(data_size > 0) {
u8 data[data_size];
if(data[0] == 0x42 && data[1] == 0x4D) {
str suffix = ".bmp";
} else if(data[0] == 0xFF && data[1] == 0xD8 && data[2] == 0xFF && data[3] == 0xE0) {
str suffix = ".jpg";
} else if(data[0] == 0x89 && data[1] == 0x50 && data[2] == 0x4E && data[3] == 0x47) {
str suffix = ".png";
} else {
str suffix = "";
std::warning("Thumbnail file type unknown, no suffix added");
}
hex::core::add_virtual_file(std::string::to_string(name) + suffix, data);
}
padding[while(std::mem::read_unsigned($, 1) == 0x00)];
};
ThumbCacheHeader header @ 0x00;
CacheRecord records[while(!std::mem::reached(header.offset_to_first_empty))] @ header.header_size;

View File

@@ -151,8 +151,235 @@ enum TIFFTag : u16 {
YCbCrSubSampling = 0x0212,
YCbCrPositioning = 0x0213,
ReferenceBlackWhite = 0x0214,
StripRowCounts = 0x022F, // 559
XMP = 0x02BC, // 700
ImageRating = 0x4746, // 18246
ImageRatingPercent = 0x4749, // 18249
ImageID = 0x800D, // 32781
WangAnnotation = 0x80A4, // 32932
CFARepeatPatternDim = 0x828D, // 33421
CFAPattern = 0x828E, // 33422
BatteryLevel = 0x828F, // 33423
Copyright = 0x8298,
ICCProfile = 0x8773
ExposureTime = 0x829A, // 33434
FNumber = 0x829D, // 33437
MDFileTag = 0x82A5, // 33445
MDScalePixel = 0x82A6, // 33446
MDColorTable = 0x82A7, // 33447
MDLabName = 0x82A8, // 33448
MDSampleInfo = 0x82A9, // 33449
MDPrepDate = 0x82AA, // 33450
MDPrepTime = 0x82AB, // 33451
MDFileUnits = 0x82AC, // 33452
ModelPixelScaleTag = 0x830E, // 33550
IPTCNAA = 0x83BB, // 33723
INGRPacketDataTag = 0x847E, // 33918
INGRFlagRegisters = 0x847F, // 33919
IrasBTransformationMatrix = 0x8480, // 33920
ModelTiepointTag = 0x8482, // 33922
Site = 0x84E0, // 34016
ColorSequence = 0x84E1, // 34017
IT8Header = 0x84E2, // 34018
RasterPadding = 0x84E3, // 34019
BitsPerRunLength = 0x84E4, // 34020
BitsPerExtendedRunLength = 0x84E5, // 34021
ColorTable = 0x84E6, // 34022
ImageColorIndicator = 0x84E7, // 34023
BackgroundColorIndicator = 0x84E8, // 34024
ImageColorValue = 0x84E9, // 34025
BackgroundColorValue = 0x84EA, // 34026
PixelIntensityRange = 0x84EB, // 34027
TransparencyIndicator = 0x84EC, // 34028
ColorCharacterization = 0x84ED, // 34029
HCUsage = 0x84EE, // 34030
TrapIndicator = 0x84EF, // 34031
CMYKEquivalent = 0x84F0, // 34032
Reserved1 = 0x84F1, // 34033
Reserved2 = 0x84F2, // 34034
Reserved3 = 0x84F3, // 34035
ModelTransformationTag = 0x85D8, // 34264
Photoshop = 0x8649, // 34377
ExifIFD = 0x8769, // 34665
ICCProfile = 0x8773,
ImageLayer = 0x87AC, // 34732
GeoKeyDirectoryTag = 0x87AF, // 34735
GeoDoubleParamsTag = 0x87B0, // 34736
GeoAsciiParamsTag = 0x87B1, // 34737
ExposureProgram = 0x8822, // 34850
SpectralSensitivity = 0x8824, // 34852
GPSInfo = 0x8825, // 34853
ISOSpeedRatings = 0x8827, // 34855
OECF = 0x8828, // 34856
Interlace = 0x8829, // 34857
TimeZoneOffset = 0x882A, // 34858
SelfTimeMode = 0x882B, // 34859
SensitivityType = 0x8830, // 34864
StandardOutputSensitivity = 0x8831, // 34865
RecommendedExposureIndex = 0x8832, // 34866
ISOSpeed = 0x8833, // 34867
ISOSpeedLatitudeyyy = 0x8834, // 34868
ISOSpeedLatitudezzz = 0x8835, // 34869
HylaFAXFaxRecvParams = 0x885C, // 34908
HylaFAXFaxSubAddress = 0x885D, // 34909
HylaFAXFaxRecvTime = 0x885E, // 34910
ExifVersion = 0x9000, // 36864
DateTimeOriginal = 0x9003, // 36867
DateTimeDigitized = 0x9004, // 36868
ComponentsConfiguration = 0x9101, // 37121
CompressedBitsPerPixel = 0x9102, // 37122
ShutterSpeedValue = 0x9201, // 37377
ApertureValue = 0x9202, // 37378
BrightnessValue = 0x9203, // 37379
ExposureBiasValue = 0x9204, // 37380
MaxApertureValue = 0x9205, // 37381
SubjectDistance = 0x9206, // 37382
MeteringMode = 0x9207, // 37383
LightSource = 0x9208, // 37384
Flash = 0x9209, // 37385
FocalLength = 0x920A, // 37386
FlashEnergy = 0x920B, // 37387
SpatialFrequencyResponse = 0x920C, // 37388
Noise = 0x920D, // 37389
FocalPlaneXResolution = 0x920E, // 37390
FocalPlaneYResolution = 0x920F, // 37391
FocalPlaneResolutionUnit = 0x9210, // 37392
ImageNumber = 0x9211, // 37393
SecurityClassification = 0x9212, // 37394
ImageHistory = 0x9213, // 37395
SubjectLocation = 0x9214, // 37396
ExposureIndex = 0x9215, // 37397
TIFFEPStandardID = 0x9216, // 37398
SensingMethod = 0x9217, // 37399
MakerNote = 0x927C, // 37500
UserComment = 0x9286, // 37510
SubsecTime = 0x9290, // 37520
SubsecTimeOriginal = 0x9291, // 37521
SubsecTimeDigitized = 0x9292, // 37522
ImageSourceData = 0x935C, // 37724
XPTitle = 0x9C9B, // 40091
XPComment = 0x9C9C, // 40092
XPAuthor = 0x9C9D, // 40093
XPKeywords = 0x9C9E, // 40094
XPSubject = 0x9C9F, // 40095
FlashpixVersion = 0xA000, // 40960
ColorSpace = 0xA001, // 40961
PixelXDimension = 0xA002, // 40962
PixelYDimension = 0xA003, // 40963
RelatedSoundFile = 0xA004, // 40964
InteroperabilityIFD = 0xA005, // 40965
FlashEnergyExif = 0xA20B, // 41483
SpatialFrequencyResponseExif = 0xA20C,// 41484
FocalPlaneXResolutionExif = 0xA20E, // 41486
FocalPlaneYResolutionExif = 0xA20F, // 41487
FocalPlaneResolutionUnitExif = 0xA210,// 41488
SubjectLocationExif = 0xA214, // 41492
ExposureIndexExif = 0xA215, // 41493
SensingMethodExif = 0xA217, // 41495
FileSource = 0xA300, // 41728
SceneType = 0xA301, // 41729
CFAPatternExif = 0xA302, // 41730
CustomRendered = 0xA401, // 41985
ExposureMode = 0xA402, // 41986
WhiteBalance = 0xA403, // 41987
// Adobe DNG and cDNG Extensions (0xC6, 0xc7)
DNGVersion = 0xC612, // 50706
DNGBackwardVersion = 0xC613, // 50707
UniqueCameraModel = 0xC614, // 50708
LocalizedCameraModel = 0xC615, // 50709
CFALayout = 0xC617, // 50711
LinearizationTable = 0xC618, // 50712
BlackLevelRepeatDim = 0xC619, // 50713
BlackLevel = 0xC61A, // 50714
BlackLevelDeltaH = 0xC61B, // 50715
BlackLevelDeltaV = 0xC61C, // 50716
WhiteLevel = 0xC61D, // 50717
DefaultScale = 0xC61E, // 50718
DefaultCropOrigin = 0xC61F, // 50719
DefaultCropSize = 0xC620, // 50720
ColorMatrix1 = 0xC621, // 50721
ColorMatrix2 = 0xC622, // 50722
CameraCalibration1 = 0xC623, // 50723
CameraCalibration2 = 0xC624, // 50724
ReductionMatrix1 = 0xC625, // 50725
ReductionMatrix2 = 0xC626, // 50726
AnalogBalance = 0xC627, // 50727
AsShotNeutral = 0xC628, // 50728
AsShotWhiteXY = 0xC629, // 50729
BaselineExposure = 0xC62A, // 50730
BaselineNoise = 0xC62B, // 50731
BaselineSharpness = 0xC62C, // 50732
BayerGreenSplit = 0xC62D, // 50733
LinearResponseLimit = 0xC62E, // 50734
CameraSerialNumber = 0xC62F, // 50735
LensInfo = 0xC630, // 50736
ChromaBlurRadius = 0xC631, // 50737
AntiAliasStrength = 0xC632, // 50738
ShadowScale = 0xC633, // 50739
DNGPrivateData = 0xC634, // 50740
MakerNoteSafety = 0xC635, // 50741
CalibrationIlluminant1 = 0xC65A, // 50778
CalibrationIlluminant2 = 0xC65B, // 50779
BestQualityScale = 0xC65C, // 50780
RawDataUniqueID = 0xC65D, // 50781
OriginalRawFileName = 0xC68B, // 50827
OriginalRawFileData = 0xC68C, // 50828
ActiveArea = 0xC68D, // 50829
MaskedAreas = 0xC68E, // 50830
AsShotICCProfile = 0xC68F, // 50831
AsShotPreProfileMatrix = 0xC690, // 50832
CurrentICCProfile = 0xC691, // 50833
CurrentPreProfileMatrix = 0xC692, // 50834
ColorimetricReference = 0xC6BF, // 50879
CameraCalibrationSignature = 0xC6F3, // 50931
ProfileCalibrationSignature = 0xC6F4, // 50932
ExtraCameraProfiles = 0xC6F6, // 50934
AsShotProfileName = 0xC6F8, // 50936
NoiseReductionApplied = 0xC6F9, // 50937
ProfileName = 0xC6FA, // 50938
ProfileHueSatMapDims = 0xC6FB, // 50939
ProfileHueSatMapData1 = 0xC6FC, // 50940
ProfileHueSatMapData2 = 0xC6FD, // 50941
ProfileToneCurve = 0xC6FE, // 50942
ProfileEmbedPolicy = 0xC6FF, // 50943
ProfileCopyright = 0xC700, // 50944
ForwardMatrix1 = 0xC714, // 50964
ForwardMatrix2 = 0xC715, // 50965
PreviewApplicationName = 0xC716, // 50966
PreviewApplicationVersion = 0xC717, // 50967
PreviewSettingsName = 0xC718, // 50968
PreviewSettingsDigest = 0xC719, // 50969
PreviewColorSpace = 0xC71A, // 50970
PreviewDateTime = 0xC71B, // 50971
RawImageDigest = 0xC71C, // 50972
OriginalRawFileDigest = 0xC71D, // 50973
SubTileBlockSize = 0xC71E, // 50974
RowInterleaveFactor = 0xC71F, // 50975
ProfileLookTableDims = 0xC725, // 50981
ProfileLookTableData = 0xC726, // 50982
OpcodeList1 = 0xC740, // 51008
OpcodeList2 = 0xC741, // 51009
OpcodeList3 = 0xC74E, // 51022
NoiseProfile = 0xC761, // 51041
TimeCodes = 0xC763, // 51043
FrameRate = 0xc764, // 51044
TStop = 0xc772, // 51058
OriginalDefaultFinalSize = 0xC791, // 51089
OriginalBestQualityFinalSize = 0xC792,// 51090
OriginalDefaultCropSize = 0xC793, // 51091
ProfileHueSatMapEncoding = 0xC7A3, // 51107
ProfileLookTableEncoding = 0xC7A4, // 51108
BaselineExposureOffset = 0xC7A5, // 51109
DefaultBlackRender = 0xC7A6, // 51110
NewRawImageDigest = 0xC7A7, // 51111
RawToPreviewGain = 0xC7A8, // 51112
DefaultUserCrop = 0xC7B5, // 51125
DepthFormat = 0xC7E9, // 51177
DepthNear = 0xC7EA, // 51178
DepthFar = 0xC7EB, // 51179
DepthUnits = 0xC7EC, // 51180
DepthMeasureType = 0xC7ED, // 51181
EnhanceParams = 0xC7EE // 51182
};
struct ValueArray<T, auto Count> {

View File

@@ -10,6 +10,7 @@ import std.mem;
import std.string;
import std.time;
import std.io;
import type.magic;
struct TableDirectory {
char tag[4];
@@ -47,7 +48,6 @@ struct LongDatetime {
s64 time;
} [[format("format_longdatetime")]];
using HeadTable;
u64 currentOffset;
u64 currentLength;
@@ -55,7 +55,7 @@ u64 glyfStart;
bool hasPostScriptOutlines = false;
u16 numHMetrics;
u16 gNumGlyphs;
HeadTable headTable;
s16 indexToLocFormat;
// begin avar
@@ -185,12 +185,7 @@ struct EncodingRecord {
u16 encodingId;
u32 offset;
u64 endEncodingRecord = $;
$ = startOfCmapTable + offset;
CmapSubtable subtable;
$ = endEncodingRecord;
CmapSubtable subtable @ startOfCmapTable + offset;
};
struct CmapTable {
@@ -397,7 +392,7 @@ struct HeadTable {
FixedPoint32 version;
FixedPoint32 fontRevision;
u32 checksumAdjustment;
u32 magic;
type::Magic<"\x5f\x0f\x3c\xf5"> magicNumber;
HeadFlag flags;
u16 unitsPerEm;
LongDatetime created;
@@ -467,16 +462,13 @@ struct GlyfWithOffset<auto longOffset> {
u32 nextOff = u32(std::mem::read_unsigned($, 2, std::mem::Endian::Big)) * 2;
}
curGlyfSize = nextOff - realOffset;
u64 prev = $;
$ = glyfStart + realOffset;
if (curGlyfSize != 0) {
GlyfTable glyf;
GlyfTable glyf @ glyfStart + realOffset;
}
$ = prev;
};
struct LocaTable {
if (headTable.indexToLocFormat == 1) {
if (indexToLocFormat == 1) {
//u32 offsets[gNumGlyphs + 1];
GlyfWithOffset<true> glyfs[gNumGlyphs];
} else {
@@ -522,19 +514,13 @@ struct NameRecord {
u16 nameId;
u16 length;
u16 stringOffset;
u64 cur = $;
$ = curStorageOffset + stringOffset;
char16 string[length / 2];
$ = cur;
char16 string[length / 2] @ curStorageOffset + stringOffset;
};
struct LangTagRecord {
u16 length;
u16 langTagOffset;
u64 cur = $;
$ = curStorageOffset + langTagOffset;
char16 langTag[length / 2];
$ = cur;
char16 langTag[length / 2] @ curStorageOffset + langTagOffset;
};
struct NameTableV0 {
@@ -670,7 +656,7 @@ struct HiddenForPreprocessing {
("maxp"): MaxpTable table;
("head"): {
HeadTable table;
headTable = table;
indexToLocFormat = table.indexToLocFormat;
}
("glyf"): {
glyfStart = $;
@@ -716,9 +702,7 @@ struct TTF {
u16 entrySelector;
u16 rangeShift;
u64 start = $;
HiddenForPreprocessing hidden[numTables] [[hidden]];
$ = start;
HiddenForPreprocessing hidden[numTables] @ $ [[hidden]];
Table tables[numTables];
};

View File

@@ -0,0 +1,212 @@
/*
* ImHex Pattern for UEFI Firmware Volume Variable Store
*
* This file type is commonly used with virtual machine UEFI variable files, like OVMF.fd
* used with QEMU. You could also extract a UEFI firmware binary from a flash device,
* search for the FV Variable Store, and set this pattern to the FV address.
*
* A 'custom_vars.fd' can be generated with these tools:
*
* https://gitlab.com/kraxel/virt-firmware
* https://github.com/rhuefi/qemu-ovmf-secureboot/tree/master
* https://github.com/LongSoft/UEFITool
*
* 1. Generate a blank .fd file with ovmfvartool.
*
* $ ovmfvartool generate-blank blank.fd
*
* 2. Enroll the Redhat and Microsoft keys with virt-fw-vars in custom_vars.fd.
*
* $ virt-fw-vars -i blank.fd -o custom_vars.fd --enroll-redhat --secure-boot
*
* 3. Dump custom_vars.fd contents
*
* $virt-fw-vars -i custom_vars.fd -pvx
*
* or
*
* $ uefitool custom_vars.fd
*
* or use this pattern with ImHex!
*/
#pragma author Marc Jones
#pragma description UEFI Firmware Volume Variable Store
// #pragma debug
import std.core;
import std.mem;
import type.guid;
// --- GUIDs ---
#define NVRAM_FV "{FFF12B8D-7696-4C8B-A985-2747075B4F50}"
#define NVRAM_VARSTORE "{AAF32C78-947B-439A-A180-2E144EC37792}"
// --- Enumerations and Bitfields ---
// Describes the type of a file within the Firmware File System.
enum FfsFileType : u8 {
RAW = 0x01,
FREEFORM = 0x02,
SECURITY_CORE = 0x03,
PEI_CORE = 0x04,
DXE_CORE = 0x05,
PEIM = 0x06,
DRIVER = 0x07,
COMBINED_PEIM_DRIVER = 0x08,
APPLICATION = 0x09,
SMM = 0x0A,
FIRMWARE_VOLUME_IMAGE = 0x0B,
COMBINED_SMM_DXE = 0x0C,
SMM_CORE = 0x0D,
FFS_PAD = 0xF0,
};
// Attributes for a UEFI variable, indicating its properties and accessibility.
bitfield VariableAttributes{
NON_VOLATILE : 1;
BOOTSERVICE_ACCESS : 1;
RUNTIME_ACCESS : 1;
HARDWARE_ERROR_RECORD : 1;
AUTHENTICATED_WRITE_ACCESS : 1;
TIME_BASED_AUTHENTICATED_WRITE_ACCESS : 1;
APPEND_WRITE : 1;
RSVD: 25;
};
//
// Variable Store Header Format & State flags
//
enum VariableStoreFormat : u8 {
VARIABLE_STORE_FORMATTED = 0x5a,
};
enum VariableStoreState : u8 {
VARIABLE_STORE_HEALTHY = 0xfe,
};
//
// Variable State flags. See https://countchu.blogspot.com/2014/09/the-life-cycle-of-uefi-variable-in.html
//
enum VariableState : u8 {
VAR_IN_DELETED_TRANSITION = 0xfe,
VAR_DELETED = 0xfd,
VAR_HEADER_VALID_ONLY = 0x7f,
VAR_ADDED = 0x3f,
VAR_ADDED__VAR_IN_DELETED_TRANSITION__VAR_DELETED = 0x3c,
VAR_ADDED__VAR_IN_DELETED_TRANSITION = 0x3e,
VAR_ADDED__VAR_DELETED = 0x3d,
};
// --- Other Structures ---
struct EFI_TIME {
u16 Year;
u8 Month;
u8 Day;
u8 Hour;
u8 Minute;
u8 Second;
u8 Pad1;
u32 Nanosecond;
u16 TimeZone;
u8 Daylight;
u8 Pad2;
};
// --- Firmware Volume Structures ---
// Header for a block in the firmware volume map.
struct EFI_FV_BLOCK_MAP_ENTRY {
u32 NumBlocks;
u32 Length;
};
// The main header for a Firmware Volume.
struct EFI_FIRMWARE_VOLUME_HEADER {
u128 ZeroVector;
type::GUID FileSystemGuid;
u64 FvLength;
u32 Signature;
u32 Attributes;
u16 HeaderLength;
u16 Checksum;
u16 ExtHeaderOffset;
u8 Reserved;
u8 Revision;
EFI_FV_BLOCK_MAP_ENTRY BlockMap[while(std::mem::read_unsigned($, 4) != 0 || std::mem::read_unsigned($ + 4, 4) != 0)];
EFI_FV_BLOCK_MAP_ENTRY BlockMapTerminator; // After the loop, explicitly parse the (0,0) terminator entry
}[[single_color]];
// --- UEFI Variable Structures ---
struct VARIABLE_STORE_HEADER {
type::GUID Signature;
u32 Size;
VariableStoreFormat Format;
VariableStoreState State;
u16 Reserved;
u32 Reserved1;
}[[single_color]];
#define VAR_START_ID 0x55AA
struct VARIABLE_HEADER {
u16 StartId;
VariableState State;
u8 Reserved;
VariableAttributes Attributes;
u32 NameSize;
u32 DataSize;
type::GUID VendorGuid;
};
struct AUTHENTICATED_VARIABLE_HEADER {
u16 StartId;
VariableState State;
u8 Reserved;
VariableAttributes Attributes;
u64 MonotonicCount;
EFI_TIME TimeStamp;
u32 PubKeyIndex;
u32 NameSize;
u32 DataSize;
type::GUID VendorGuid;
};
struct UEFI_VARIABLE {
AUTHENTICATED_VARIABLE_HEADER Header; // TODO: Check authenticated vs normal variable...
char16 Name[Header.NameSize / 2]; // Name is a UTF-16 string
u8 Data[Header.DataSize];
// Align to the next 4-byte boundary for the next variable.
u8 pad[std::mem::align_to(4, sizeof(this)) - sizeof(this)];
} [[name(this.Name), single_color]];
// --- Main Pattern Entry Point ---
EFI_FIRMWARE_VOLUME_HEADER FV_Header @ 0;
if (std::core::formatted_value(FV_Header.FileSystemGuid) != "{FFF12B8D-7696-4C8B-A985-2747075B4F50}") {
std::error(std::format("Unknown FV_Header.FileSystemGuid {}", std::core::formatted_value(FV_Header.FileSystemGuid)));
}
// The next structure should be the Variable Store Header
VARIABLE_STORE_HEADER VarStore @ $;
if (std::core::formatted_value(VarStore.Signature) != NVRAM_VARSTORE) {
std::error(std::format("Unknown VarStore.Signature {}", std::core::formatted_value(VarStore.Signature)));
}
// Index through the Uefi variables until we don't find a Variable Signature 0x55AA
UEFI_VARIABLE UefiVars[while(std::mem::read_unsigned($, 2) == VAR_START_ID)] @ $;
// TODO: grey out the Uefi variables that are in the non-active state, != VAR_ADDED.

View File

@@ -1,5 +1,6 @@
#pragma author WerWolv
#pragma description USB Flashing Format (.uf2)
#pragma magic [ 55 46 32 0A 57 51 5D 9E ] @ 0x00
import std.sys;
import std.mem;

156
patterns/upk-ue3.hexpat Normal file
View File

@@ -0,0 +1,156 @@
#pragma description Unreal Engine 3 UPK
#pragma endian little
#pragma magic [ c1 83 2a 9e ] @ 0x00
import std.io;
#define HEADER_LOCATION 0x00
#define FILE_SIGNATURE 0x9E2A83C1
#define VER_GIGANTIC 867
#define VER_BATMAN_CITY 805
using offset = u32;
bool isCompressed = false;
// Generations Info
struct FGenerationInfo {
u32 exportcount;
u32 namecount;
u32 netobjectcount;
};
// Compressed Chunk Info
struct CompressedChunkInfo {
offset uncompressedoffset;
u32 uncompressedsize;
offset compressedoffset;
u32 compressedsize;
};
// Name Table Entry
struct FNameEntry {
s32 namelength;
char name[];
u64 nameflags;
};
// Import Table Entry
struct FObjectImport {
u64 packageid;
u64 typeid;
u32 ownerref;
u64 nameid;
};
// Export Table Entry
struct FObjectExport {
u32 typeref;
u32 parentclassref;
u32 ownerref;
u64 nameid;
u32 archetyperef;
u32 objectflagsh;
u32 objectflagsl;
u32 serialsize;
offset serialoffset;
u32 exportflags;
u32 netobjectcount;
u128 guid;
u32 unknown1;
u8 unknown2[netobjectcount * 4];
};
// Main File Header
struct FileHeader {
// File signature
u32 signature;
// Version
u16 version;
// Licensee
u16 licensee;
// Header Size
u32 headersize;
// FolderName
u32 foldername ;
// PackageInfo
char packagegroup[];
u32 packageflags; // todo: properly expand this
// GigUnknown
if (version == VER_GIGANTIC) {
u32 gigunknown1; // this seems to be unique to gig's upk format
}
// Name Table Info
u32 namecount;
offset nameoffset;
// Export Table Info
u32 exportcount;
offset exportoffset;
// Import Table Info
u32 importcount;
offset importoffset;
// Depends Info
offset dependsoffset;
// Unknown
if (version >= VER_BATMAN_CITY) {
offset serialoffset;
u32 ueunknown1; // these are present in other ue3 upks
u32 ueunknown2;
u32 ueunknown3;
}
// GUID
u128 packageguid;
// Generations Info
u32 generationscount;
FGenerationInfo generations[generationscount];
// Engine Info
u32 engineversion;
u32 cookerversion;
// Compression Info
u32 compressionflags;
u32 compressedchunks;
CompressedChunkInfo chunks[compressedchunks];
};
// Print Info
fn print_info(FileHeader header) {
if(header.signature == FILE_SIGNATURE){
std::print("UPK: Yes");
} else {
std::print("UPK: No");
}
std::print("Version: {} ",header.version);
std::print("Licensee: {} ",header.licensee);
std::print("Compressed: {} ",isCompressed);
std::print("Name Count: {} ",header.namecount);
std::print("Import Count: {} ",header.importcount);
std::print("Export Count: {} ",header.exportcount);
};
FileHeader header @ HEADER_LOCATION;
// Check Compression
if (header.compressionflags != 0 || header.compressedchunks > 0) {
isCompressed = true;
print_info(header);
return;
}
FNameEntry NameTable[header.namecount] @ header.nameoffset;
FObjectImport ImportTable[header.importcount] @ header.importoffset;
FObjectExport ExportTable[header.exportcount] @ header.exportoffset;
print_info(header);

62
patterns/valve_vpk.hexpat Normal file
View File

@@ -0,0 +1,62 @@
#pragma author Enaium
#pragma description Valve VPK
import std.io;
import std.mem;
import std.string;
struct string {
char text[while(std::mem::read_unsigned($, 1) != 0x0)];
u8 end [[hidden]];
if (text == "") {
break;
}
};
struct entry {
string name;
u32 crc;
u16 preload_bytes;
u16 archive_index;
u32 entry_offset;
u32 entry_length;
u16 terminator;
if (preload_bytes > 0x0) {
u8 preload[preload_bytes];
}
if (archive_index == 0x7fff) {
u8 data[entry_length] @ entry_offset;
}
};
struct path {
string name;
entry entry[while(true)];
};
struct extension {
string name;
path path[while(true)];
};
struct header {
u32 signature;
u32 version;
u32 tree_size;
if (version == 0x2) {
u32 file_data_section_size;
u32 archive_md5_section_size;
u32 ohter_md5_section_size;
u32 signature_section_size;
}
};
struct vpk {
header header;
extension extension[while(true)];
};
vpk vpk @ 0x0;

270
patterns/vox.hexpat Normal file
View File

@@ -0,0 +1,270 @@
#pragma author Jake Ryan
#pragma description MagicaVoxel scene description format
#pragma magic [ 56 4F 58 20 ] @ 0
#pragma array_limit 0x100000
import std.io;
import std.core;
// Input and output variables
// If true, color data will be formatted as normalized floats.
bool formatColorsAsFloat in;
// If true, chunks will be treated as though they were in a single big array instead of a hierarchy.
bool readChunksAsLinear in;
bool attemptRecoveryOnChunkSizeMismatch in;
// Pattern definitions
// https://github.com/ephtracy/voxel-model/blob/master/MagicaVoxel-file-format-vox.txt
// https://github.com/ephtracy/voxel-model/blob/master/MagicaVoxel-file-format-vox-extension.txt
struct VoxHeader
{
char magic[4];
u32 version;
};
struct Voxel
{
u8 x;
u8 y;
u8 z;
u8 colorIndex;
} [[static, format("FormatVoxel")]];
fn FormatVoxel(ref Voxel v)
{
return std::format("{}, {}, {}", v.x, v.y, v.z);
};
struct RGBA8
{
u8 r [[color("FF0000")]];
u8 g [[color("00FF00")]];
u8 b [[color("0000FF")]];
u8 a;
} [[static, color(std::format("{:02X}{:02X}{:02X}", r, g, b)), format("FormatRGBA8")]];
fn FormatRGBA8(ref RGBA8 c)
{
if (formatColorsAsFloat)
{
return std::format("{:4.2f}, {:4.2f}, {:4.2f}, {:4.2f}", c.r / 255.0, c.g / 255.0, c.b / 255.0, c.a / 255.0);
}
else
{
return std::format("{:3}, {:3}, {:3}, {:3}", c.r, c.g, c.b, c.a);
}
};
struct String
{
u32 sizeBytes;
char data[sizeBytes];
} [[format("FormatString")]];
fn FormatString(ref String s)
{
return std::format("\"{}\"", s.data);
};
struct Vec3 { float v[3]; };
struct Mat3 { Vec3 r0, r1, r2; } [[format("FormatMat3")]];
fn FormatMat3(ref Mat3 m)
{
return std::format("({}, {}, {}) ({}, {}, {}) ({}, {}, {})",
m.r0.v[0], m.r0.v[1], m.r0.v[2],
m.r1.v[0], m.r1.v[1], m.r1.v[2],
m.r2.v[0], m.r2.v[1], m.r2.v[2]);
};
bitfield Rotation
{
firstRowIndex : 2;
secondRowIndex : 2;
firstRowSign : 1;
secondRowSign : 1;
thirdRowSign : 1;
padding : 1;
};
// Unused, but demonstrates how a 3x3 matrix could be constructed from Rotation.
fn RotationToMat3(Rotation r)
{
Mat3 mat;
mat.r0.v[r.firstRowIndex] = -(r.firstRowSign * 2 - 1);
mat.r1.v[r.secondRowIndex] = -(r.secondRowSign * 2 - 1);
mat.r2.v[0] = -(r.thirdRowSign * 2.0 - 1) * !(r.firstRowIndex == 0 || r.secondRowIndex == 0);
mat.r2.v[1] = -(r.thirdRowSign * 2.0 - 1) * !(r.firstRowIndex == 1 || r.secondRowIndex == 1);
mat.r2.v[2] = -(r.thirdRowSign * 2.0 - 1) * !(r.firstRowIndex == 2 || r.secondRowIndex == 2);
return mat;
};
struct KeyValue
{
String key;
// Depending on the key, the value must be parsed as one or more floats or integers.
String value;
} [[format("FormatKeyValue")]];
fn FormatKeyValue(ref KeyValue kv)
{
return std::format("{}: {}", kv.key, kv.value);
};
struct Dict
{
u32 count;
KeyValue pairs[count] [[inline]];
} [[format("FormatDict")]];
fn FormatDict(ref Dict d)
{
return std::format("count: {}", d.count);
};
struct Model
{
s32 modelId;
Dict attributes;
};
struct Chunk
{
char id[4];
u32 sizeBytes;
u32 childrenBytes;
// Contents of the chunk
s32 cursorPosBeforeContent = $;
match (id)
{
("MAIN"): {}
("PACK"):
{
u32 numModels;
}
("SIZE"):
{
u32 sizeX, sizeY, sizeZ;
}
("XYZI"):
{
u32 numVoxels;
Voxel voxels[numVoxels];
}
("RGBA"):
{
RGBA8 colors[256];
}
("nTRN"):
{
s32 nodeId;
Dict attributes;
s32 childNodeId;
s32 reservedMustBeNegative1;
s32 layerId;
u32 numFrames;
Dict frames[numFrames];
}
("nGRP"):
{
s32 nodeId;
Dict attributes;
u32 numChildren;
s32 childrenIds[numChildren];
}
("nSHP"):
{
s32 nodeId;
Dict attributes;
u32 numModels;
Model models[numModels];
}
("MATL"): // Material properties
{
s32 materialId;
Dict attributes;
}
("LAYR"):
{
s32 layerId;
Dict attributes;
s32 reservedMustBeNegative1;
}
("rOBJ"): // Rendering attributes
{
Dict attributes;
}
("rCAM"): // Camera attributes
{
s32 cameraId;
Dict attributes;
}
("NOTE"): // Names for colors
{
u32 numNames;
String names[numNames];
}
("IMAP"): // Index map
{
// The documentation says this field is i32x256, but it's u8x256 in actual models.
u8 paletteIndices[256];
}
(_):
{
u8 unknownData[sizeBytes];
std::warning(std::format("Unknown chunk ID at 0x{:X}: \"{}\"", $, id));
}
}
s32 actualContentSize = ($ - cursorPosBeforeContent);
if (actualContentSize != sizeBytes)
{
str warningTextSecondHalf = std::format("Content size mismatch! Expected: {}. Actual: {}", sizeBytes, actualContentSize);
str warningText = std::format("Chunk at 0x{:X} with id {}: {}", addressof(this), id, warningTextSecondHalf);
if (attemptRecoveryOnChunkSizeMismatch)
{
std::warning(warningText);
}
else
{
std::error(warningText);
}
// Limited recovery- reading past EoF is still possible if chunk is nested.
// Setting readChunksAsLinear to true can make recovery more robust as the outermost array will be broken out of.
break;
}
if (childrenBytes > 0 && !readChunksAsLinear)
{
s32 cursorPosBeforeChildren = $;
Chunk children[while(true)];
}
// The node with id MAIN is also the root node.
if (id != "MAIN" && !readChunksAsLinear)
{
// We are done parsing the children for this parent if the cursor has advanced parent.childrenBytes bytes since then.
if ($ - parent.cursorPosBeforeChildren >= parent.childrenBytes)
{
break;
}
}
} [[format("ChunkFormatter")]];
fn ChunkFormatter(ref Chunk chunk)
{
return chunk.id;
};
VoxHeader header @ 0 [[inline]];
Chunk mainChunk[readChunksAsLinear ? 0 : 1] @ $ [[inline]];
Chunk chunks[while(readChunksAsLinear && !std::mem::eof())] @ $;
if (std::core::member_count(mainChunk) > 0)
{
std::core::set_display_name(mainChunk[0], "mainChunk");
}

View File

@@ -7,8 +7,8 @@ import type.size;
struct Sound {
char name[];
u32 length;
u8 data[length];
};
u8 data[length] [[sealed]];
} [[name(name), hex::visualize("sound", data, 1, 8000)]];
struct Header {
type::Size<u32> size;
@@ -21,4 +21,4 @@ struct WAS {
Sound sounds[header.count];
};
WAS was @ 0x00;
WAS was @ 0x00;

View File

@@ -24,6 +24,7 @@
#include "std/mem.pat"
#include "std/string.pat"
using WBTLine;
using WBTTimestamp;
// helper to format latitude and longitude in a human-readable form
@@ -43,6 +44,11 @@ fn iso8601(WBTTimestamp input) {
};
// short format for overview in a list
fn shortString(WBTLine input) {
return std::format("{:02d}:{:02d}:{:02d} {:s},{:s}", input.timestamp.hours, input.timestamp.minutes, input.timestamp.seconds, stringDegrees(input.lat), stringDegrees(input.lon));
};
// This 16 bit field is barely used. Only these two are known.
bitfield WBTFlags {
split: 1 [[comment("start of a new file")]];
@@ -70,7 +76,7 @@ struct WBTLine {
s32 lat [[color("007FFF"), format("stringDegrees")]];
s32 lon [[color("7F00FF"), format("stringDegrees")]];
s16 alt [[color("0000FF")]];
} [[hex::visualize("coordinates", decimalDegrees(lat), decimalDegrees(lon))]];
} [[hex::visualize("coordinates", decimalDegrees(lat), decimalDegrees(lon)), format("shortString")]];
// parsing that whole file start to finish:

79
patterns/zim.hexpat Normal file
View File

@@ -0,0 +1,79 @@
#pragma endian little
#pragma description ZIM file format
#pragma magic [ 5A 49 4D 04 ] @ 0x00
u32 maxEntries in;
import std.string;
import std.mem;
import std.math;
import std.core;
struct Header {
char signature[4];
u16 majorVersion;
u16 minorVersion;
u128 uid;
u32 entryCount, clusterCount;
u64 pathPtrPos, titlePtrPos, clusterPtrPos, mimeListPos;
u32 mainPage;
u32 layoutPage;
u64 checksumPos;
};
Header header @ 0x00;
std::string::NullString mimeTypes[while(std::mem::read_unsigned($, 1) != 0x00)] @ header.mimeListPos;
fn mimename(u16 index) {
if(index == 0xFFFF) {
return "redirect";
} else {
return mimeTypes[index];
}
};
enum CompressionType : u8 {
None = 0x01,
LZMA2 = 0x04,
zstd = 0x05
};
bitfield ClusterInfo {
CompressionType compression: 4;
offsetSize: 1;
} [[bitfield_order(std::core::BitfieldOrder::LeastToMostSignificant, 8)]];
struct Cluster {
ClusterInfo info;
} [[inline]];
struct ClusterPointer {
Cluster *cluster: u64;
} [[inline]];
struct Entry {
u16 mimetype [[format("mimename")]];
u8 parameterlen [[hidden]];
char _namespace [[name("namespace")]];
u32 revision;
if (mimetype == 0xFFFF) {
u32 redirectIndex;
} else {
u32 clusterNumber;
ClusterPointer cluster @ (header.clusterPtrPos + clusterNumber * 8);
u32 blobNumber;
}
std::string::NullString path;
std::string::NullString title;
padding[parameterlen];
} [[inline]];
struct EntryPointer {
Entry *entry : u64;
} [[inline]];
EntryPointer pathPointerList[std::math::min(header.entryCount, maxEntries == 0 ? 1500 : maxEntries)] @ header.pathPtrPos;
u32 titlePointerList[header.entryCount] @ header.titlePtrPos;

View File

@@ -4,43 +4,65 @@
import std.mem;
import std.math;
import std.core;
import type.time;
struct EndOfCentralDirectory {
u32 headerSignature [[comment("EoCD magic"), name("EoCD PK\\5\\6")]];
u16 diskNum [[comment("Number of this disk "), name("Disk Number")]];
u16 diskStart [[comment("Disk where central directory starts "), name("Central Directory Disk Number")]];
u16 CDRCount [[comment("Number of central directory records on this disk"), name("Central Directory Entries")]];
u16 CentralDirectoryRecordCount [[comment("Total number of entries in the central directory"), name("Total Central Directory Entries")]];
u32 CDSize [[comment("Size of central directory (bytes)"), name("Central Directory Size")]];
u32 CDOffset [[comment("Offset of start of central directory, relative to start of archive"), name("Central Directory Offset")]];
u16 commentLength [[color("00000000")]];
char coment[commentLength] [[name("Comment")]];
using CentralDirectoryFileHeader;
struct EOCD64Locator {
u32 headerSignature [[comment("EoCD magic"), name("EoCD PK\\6\\7")]];
u32 cdrDisk [[comment("Disk number containing the end of central directory record"), name("CDR Disk")]];
u64 eocdOffset [[comment("Offset of end of central directory record"), name("End of Central Directory Record Offset")]];
u32 totalDisks [[comment("Total number of disks"), name("Total Disks")]];
};
struct EndOfCentralDirectory {
u32 magic;
if (magic == 0x06064b50) {
u64 eocdSize [[comment("Size of fixed fields + size of variable data - 12"), name("EOCD Size")]];
u16 madeByVersion [[comment("The version of zip this was authored by"), name("Made By Version")]];
u16 versionNeeded [[comment("The minimum supported ZIP version needed to extract the file"), name("Version Needed")]];
u32 diskNum [[comment("number of this disk"), name("Disk Number")]];
u32 diskStart [[comment("Disk where central directory starts "), name("Central Directory Disk Number")]];
u64 CDRCount [[comment("Number of central directory records on this disk"), name("Central Directory Entries")]];
u64 CentralDirectoryRecordCount [[comment("Total number of entries in the central directory"), name("Total Central Directory Entries")]];
u64 CDSize [[comment("Size of central directory (bytes)"), name("Central Directory Size")]];
u64 CDOffset [[comment("Offset of start of central directory, relative to start of archive"), name("Central Directory Offset")]];
char extra[eocdSize - 44] [[comment("zip64 extensible data sector"), name("Extra Data")]];
EOCD64Locator locator [[name("EOCD Locator")]];
char eocd32[20] [[name("EOCD32")]];
u16 commentLength [[color("00000000")]];
char coment[commentLength] [[name("Comment")]];
CentralDirectoryFileHeader centralDirHeaders[CDRCount] @ (CDOffset) [[name("Files")]];
} else if (magic == 0x06054B50) {
u16 diskNum [[comment("Number of this disk "), name("Disk Number")]];
u16 diskStart [[comment("Disk where central directory starts "), name("Central Directory Disk Number")]];
u16 CDRCount [[comment("Number of central directory records on this disk"), name("Central Directory Entries")]];
u16 CentralDirectoryRecordCount [[comment("Total number of entries in the central directory"), name("Total Central Directory Entries")]];
u32 CDSize [[comment("Size of central directory (bytes)"), name("Central Directory Size")]];
u32 CDOffset [[comment("Offset of start of central directory, relative to start of archive"), name("Central Directory Offset")]];
u16 commentLength [[color("00000000")]];
char coment[commentLength] [[name("Comment")]];
if (CDOffset != 0xFFFFFFFF && CentralDirectoryRecordCount != 0xFFFF) {
CentralDirectoryFileHeader centralDirHeaders[CDRCount] @ (CDOffset) [[name("Files")]];
}
} else {
std::error("Invalid EOCD magic!");
}
};
namespace extra {
bitfield Flags {
bitfield UTFlags {
bool modification_time_set : 1;
bool access_time_set : 1;
bool creation_time_set : 1;
reserved: 5; //reserved for additional timestamps; not set
};
struct X5455_ExtendedTimestamp {
Flags Flags;
if (Flags.modification_time_set){
u32 ModTime;
}
if (Flags.access_time_set){
u32 AcTime;
}
if (Flags.creation_time_set){
u32 CrTime;
}
};
struct X000A_NTFS {
u32 reserved;
u16 tag;
@@ -72,58 +94,97 @@ namespace extra {
}
};
struct ZIP64_SizeInfo {
u64 uncompressedSize;
u64 compressedSize;
if (parent.TSize > 16){
u64 localHeaderOffset;;
}
if (parent.TSize > 24){
u32 diskStartNumber;
}
};
struct ExtraField {
u16 tag;
u16 TSize;
if (tag == 0x5455){
extra::X5455_ExtendedTimestamp x5455_ExtendedTimestamp;
}else if (tag == 0x000a){
if (tag == 0x0001) {
extra::ZIP64_SizeInfo ZIP64_SizeInfo;
} else if (tag == 0x5455) {
// 0x5455 needs parsed with TSize in context to prevent overshoot from mismatched TSize/flags set
UTFlags Flags;
u64 extraEndFromFlags = $ + 4*(Flags.modification_time_set + Flags.access_time_set + Flags.creation_time_set);
u64 extraEndFromTSize = $ + TSize - 1;
u64 extraEnd = std::math::min(extraEndFromFlags, extraEndFromTSize);
if ( ($ < extraEnd) && Flags.modification_time_set) {
type::time32_t ModTime;
}
if ( ($ < extraEnd) && Flags.access_time_set) {
type::time32_t AcTime;
}
if ( ($ < extraEnd) && Flags.creation_time_set) {
type::time32_t CrTime;
}
} else if (tag == 0x000a) {
extra::X000A_NTFS x000A_NTFS;
}else if (tag == 0x7875){
} else if (tag == 0x7875) {
extra::X7875_NewUnix x7875_NewUnix;
}else if (tag == 0x5855){
} else if (tag == 0x5855) {
extra::X5855_InfoZipUnix x5855_InfoZipUnix;
}else{
} else {
std::print("Unsupported tag 0x{:02X}", tag);
padding[TSize];
}
};
fn has_extra_field(u32 extraEnd) {
if ($ + 4 > extraEnd) return false;
u16 tag = std::mem::read_unsigned($, 2, std::mem::Endian::Little);
u16 len = std::mem::read_unsigned($ + 2, 2, std::mem::Endian::Little);
return !(tag == 0 || len == 0) && $ + 4 + len <= extraEnd;
};
}
fn find_eocd() {
// If there is no zip comment, which is the common case,
// the end-of-central-directory record will be 22 bytes long
// at the end of the file; check if size-22 has the signature.
if (std::mem::read_unsigned(std::mem::size()-22, 4, std::mem::Endian::Little) == 0x06054B50) {
return std::mem::size()-22;
} else {
// If it's not there, then there's probably a zip comment;
// search the last 64KB of the file for the signature.
u128 offset_search_from = std::math::max(0, std::mem::size()-65536-22);
u128 prev_address;
while(1){
u128 current_address = std::mem::find_sequence_in_range(0, offset_search_from, std::mem::size(), 0x50,0x4B,0x05,0x06);
u128 offset_search_from = std::math::max(0, std::mem::size()-65536);
u128 prev_address;
while(1){
s128 current_address = std::mem::find_sequence_in_range(0, offset_search_from, std::mem::size(), 0x50,0x4B,0x05,0x06);
//Reached EOF and did not find valid eocd.
if (current_address == 340282366920938463463374607431768211455){
std::error("Could not find EOCD.");
//Reached EOF and did not find valid eocd.
if (current_address == -1) {
std::error("Could not find EOCD.");
}
//Potential eocd found. Create a eocd struct
EndOfCentralDirectory EOCD32 @ current_address;
if (EOCD32.CDOffset == 0xFFFFFFFF || EOCD32.CentralDirectoryRecordCount == 0xFFFF) {
// this is a zip64 file
if (std::mem::read_unsigned(current_address - 20, 4, std::mem::Endian::Little) == 0x07064B50){
EOCD64Locator locator @ current_address - 20;
EndOfCentralDirectory EOCD64 @ locator.eocdOffset;
//If central directory file header is valid, then we know the eocd offset is valid.
if (std::mem::read_unsigned(EOCD64.CDOffset, 4, std::mem::Endian::Little) == 0x2014B50){
return locator.eocdOffset;
}
}
//Potential eocd found. Create a eocd struct
EndOfCentralDirectory EOCD @ current_address;
} else {
//If central directory file header is valid, then we know the eocd offset is valid.
if (std::mem::read_unsigned(EOCD.CDOffset, 4, std::mem::Endian::Little) == 0x2014B50){
if (std::mem::read_unsigned(EOCD32.CDOffset, 4, std::mem::Endian::Little) == 0x2014B50){
return current_address;
}
offset_search_from = current_address + 1;
prev_address = current_address;
}
offset_search_from = current_address + 1;
prev_address = current_address;
}
};
EndOfCentralDirectory fileInfo @ find_eocd() [[name("End of Central Directory Record")]];
enum CompressionMethod : u16 {
None = 0, // The file is stored (no compression)
@@ -170,13 +231,14 @@ bitfield GeneralPurposeBitFlags {
reservedPKWARE_1 : 2;
};
struct LocalFileHeader {
u32 headerSignature [[name("LCF PK\\3\\4")]];
u16 version [[ comment("The minimum supported ZIP specification version needed to extract the file") ]];
GeneralPurposeBitFlags generalPurposeBitFlags [[ comment("General purpose bit flag") ]];
CompressionMethod compressionMethod [[ comment("Compression method") ]];
u16 lastModifyTime [[ comment("File last modification time") ]];
u16 lastModifyDate [[ comment("File last modification date") ]];
type::DOSTime lastModifyTime [[ comment("File last modification time") ]];
type::DOSDate lastModifyDate [[ comment("File last modification date") ]];
u32 crc32 [[ comment("CRC-32") ]];
u32 compressedSize [[ comment("Compressed size") ]];
u32 uncompressedSize [[ comment("Uncompressed size") ]];
@@ -184,13 +246,34 @@ struct LocalFileHeader {
u16 extraFieldLength [[ comment("Extra field length (m)") ]];
char fileName[fileNameLength] [[ comment("File Name") ]];
u64 extraEnd = $ + extraFieldLength;
extra::ExtraField extraFields[while ($ < extraEnd)] [[comment("Extra Fields")]];
u8 data[compressedSize] [[name("File Data")]];
extra::ExtraField extraFields[while (extra::has_extra_field(extraEnd))] [[comment("Extra Fields")]];
padding[extraEnd - $];
u8 data[get_file_data_size(compressionMethod, compressedSize, uncompressedSize, extraFields)] [[name("File Data")]];
};
union File {
u32 fileOffset [[comment("Offset of local file header, relative to the start of the first disk on which the file occurs.")]];
LocalFileHeader *fileHeader : u32;
fn get_file_data_size(CompressionMethod compressionMethod, u32 compressedSize, u32 uncompressedSize, ref extra::ExtraField extraFields) {
u32 size = 0;
if (compressionMethod == CompressionMethod::None) {
size = uncompressedSize;
} else {
size = compressedSize;
}
if (size != 0xFFFFFFFF) {
return size;
}
u32 extraSize = std::core::member_count(extraFields);
for (u32 i = 0, i < extraSize, i += 1) {
if (extraFields[i].tag == 0x0001) {
if (compressionMethod == CompressionMethod::None) {
return extraFields[i].ZIP64_SizeInfo.uncompressedSize;
} else {
return extraFields[i].ZIP64_SizeInfo.compressedSize;
}
}
}
return 0;
};
struct CentralDirectoryFileHeader {
@@ -199,8 +282,8 @@ struct CentralDirectoryFileHeader {
u16 versionExtract [[comment("Minimum version needed to extract")]];
GeneralPurposeBitFlags generalPurposeBitFlags [[comment("General purpose bit flag")]];
CompressionMethod compressionMethod;
u16 fileLastModifyTime [[comment("File last modification time")]];
u16 fileLastModifyDate [[comment("File last modification date")]];
type::DOSTime fileLastModifyTime [[comment("File last modification time")]];
type::DOSDate fileLastModifyDate [[comment("File last modification date")]];
u32 crc32 [[comment("CRC-32 of uncompressed data")]];
u32 compressedSize;
u32 uncompressedSize;
@@ -210,11 +293,30 @@ struct CentralDirectoryFileHeader {
u16 diskNumber [[comment("Disk number where file starts")]];
u16 internalFileAttributes;
u32 externalFileAttributes;
File file;
u32 localHeaderOffset;
char fileName[fileNameLength];
u64 extraEnd = $ + extraFieldLength;
extra::ExtraField extraFields[while ($ < extraEnd)] [[comment("Extra Fields")]];
char comment[fileCommentLength];
extra::ExtraField extraFields[while (extra::has_extra_field(extraEnd))] [[comment("Extra Fields")]];
padding[extraEnd - $];
char comment[fileCommentLength] @ extraEnd;
LocalFileHeader localFileHeader @ get_local_header_offset(localHeaderOffset, extraFields) [[name("Local File Header")]];
};
CentralDirectoryFileHeader centralDirHeaders[fileInfo.CDRCount] @ (fileInfo.CDOffset) [[name("Files")]];
fn get_local_header_offset(u32 localHeaderOffset, ref extra::ExtraField extraFields) {
u32 size = 0;
if (localHeaderOffset != 0xFFFFFFFF) {
return localHeaderOffset;
}
u32 extraSize = std::core::member_count(extraFields);
for (u32 i = 0, i < extraSize, i += 1) {
if (extraFields[i].tag == 0x0001) {
return extraFields[i].ZIP64_SizeInfo.localHeaderOffset;
}
}
std::error("No valid local header offset found!");
};
EndOfCentralDirectory fileInfo @ find_eocd() [[name("End of Central Directory Record")]];

View File

@@ -6,8 +6,29 @@
#include <wolv/utils/string.hpp>
#include <cstdlib>
#include <pl/patterns/pattern.hpp>
#define EXIT_SKIP 77
bool validatePatternValues(pl::ptrn::Pattern *pattern) {
auto value = pattern->getFormattedValue();
if (!pattern->hasValidFormattedValue()) {
fmt::print("Invalid formatted value \"{}\" of pattern \"{}\"\n", value, pattern->getDisplayName());
return false;
}
for (const auto &[address, child] : pattern->getChildren()) {
if (pattern == child)
continue;
child->setOffset(address);
if (!validatePatternValues(child))
return false;
}
return true;
}
int main(int argc, char **argv) {
// Any number of arguments other than 5 are invalid
if (argc != 4 && argc != 5)
@@ -51,7 +72,7 @@ int main(int argc, char **argv) {
}
runtime.setDangerousFunctionCallHandler([]{ return true; });
runtime.setIncludePaths({ includePath });
runtime.setIncludePaths({ includePath, patternFilePath.parent_path() });
runtime.addPragma("MIME", DummyPragmaHandler);
runtime.addPragma("description", DescPragmaHandler);
runtime.addDefine("__PL_UNIT_TESTS__");
@@ -87,6 +108,17 @@ int main(int argc, char **argv) {
return EXIT_FAILURE;
}
for (const auto &pattern : runtime.getPatterns()) {
if (!validatePatternValues(pattern.get()))
return EXIT_FAILURE;
}
for (const auto &[id, section] : runtime.getSections()) {
for (const auto &pattern : runtime.getPatterns(id)) {
if (!validatePatternValues(pattern.get()))
return EXIT_FAILURE;
}
}
} else {
// Parse the file
fmt::println("Parsing pattern {} without executing it", patternName);

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

345
themes/Nocturne.json Normal file
View File

@@ -0,0 +1,345 @@
{
"base": "Dark",
"name": "Nocturne",
"image_theme": "dark",
"authors": "JeanxPereira",
"description": "minimal dark theme based on Dracula but more darker",
"colors": {
"imgui": {
"border": "#3B3B487F",
"border-shadow": "#00000000",
"button": "#BB96F688",
"button-active": "#BB96F6FF",
"button-hovered": "#BB96F6FF",
"check-mark": "#BB96F6FF",
"child-background": "#00000000",
"docking-empty-background": "#0F0F0FEF",
"docking-preview": "#4296F9B2",
"drag-drop-target": "#FFFF00E5",
"frame-background": "#BB96F633",
"frame-background-active": "#BB96F687",
"frame-background-hovered": "#BB96F674",
"header": "#BB96F66B",
"header-active": "#BB96F6FF",
"header-hovered": "#B18DEAD0",
"menu-bar-background": "#1C1C1CFF",
"modal-window-dim-background": "#CCCCCC30",
"nav-highlight": "#BB96F6FF",
"nav-windowing-background": "#CCCCCC33",
"nav-windowing-highlight": "#FFFFFFB2",
"plot-histogram": "#E5B200FF",
"plot-histogram-hovered": "#FF9900FF",
"plot-lines": "#9B9B9BFF",
"plot-lines-hovered": "#FF6D59FF",
"popup-background": "#141414FF",
"resize-grip": "#BB96F660",
"resize-grip-active": "#BB96F6EF",
"resize-grip-hovered": "#C2A0F7D3",
"scrollbar-background": "#05050587",
"scrollbar-grab": "#313131FF",
"scrollbar-grab-active": "#828282FF",
"scrollbar-grab-hovered": "#686868FF",
"separator": "#3B3B487F",
"separator-active": "#BB96F6FF",
"separator-hovered": "#BB96F6D0",
"slider-grab": "#BB96F6FF",
"slider-grab-active": "#BB96F6FF",
"tab": "#4B405CCB",
"tab-active": "#BB96F6AE",
"tab-hovered": "#C5A5FACC",
"tab-unfocused": "#36313FFF",
"tab-unfocused-active": "#BB96F69B",
"table-border-light": "#323235FF",
"table-border-strong": "#26262BFF",
"table-header-background": "#303033FF",
"table-row-background": "#00000000",
"table-row-background-alt": "#FFFFFF0F",
"text": "#EAEAEAFF",
"text-disabled": "#7F7F7FFF",
"text-selected-background": "#BB96F675",
"title-background": "#232323FF",
"title-background-active": "#232323FF",
"title-background-collapse": "#232323FF",
"window-background": "#111111FF",
"window-shadow": "#000000FF"
},
"imhex": {
"IEEE-tool-exp": "#5D7F5DFF",
"IEEE-tool-mantissa": "#7F5D5DFF",
"IEEE-tool-sign": "#5D5D7FFF",
"achievement-unlocked": "#F1C40FFF",
"advanced-encoding-ascii": "#06539BFF",
"advanced-encoding-multi": "#F1C40FFF",
"advanced-encoding-single": "#E74C3CFF",
"advanced-encoding-unknown": "#E74C3CFF",
"blur-background": "#00000000",
"desc-button": "#141414FF",
"desc-button-active": "#3C3C3CFF",
"desc-button-hovered": "#282828FF",
"diff-added": "#388B42FF",
"diff-changed": "#F1C40FFF",
"diff-removed": "#E74C3CFF",
"find-highlight": "#672A78FF",
"highlight": "#4DC69BFF",
"logger-debug": "#388B42FF",
"logger-error": "#E74C3CFF",
"logger-fatal": "#672A78FF",
"logger-info": "#06539BFF",
"logger-warning": "#F1C40FFF",
"patches": "#E74C3CFF",
"pattern-selected": "#3683CBFF",
"toolbar-blue": "#267BC9FF",
"toolbar-brown": "#DBB377FF",
"toolbar-gray": "#E6E6E6FF",
"toolbar-green": "#59B364FF",
"toolbar-purple": "#672A78FF",
"toolbar-red": "#E74C3CFF",
"toolbar-yellow": "#F1C40FFF"
},
"imnodes": {
"box-selector": "#BB96F655",
"box-selector-outline": "#BB96F6B6",
"grid-background": "#282832C8",
"grid-line": "#C8C8C828",
"grid-line-primary": "#F0F0F03C",
"link": "#BB96F6C2",
"link-hovered": "#CEB3FAFF",
"link-selected": "#BB96F6FF",
"mini-map-background": "#19191996",
"mini-map-background-hovered": "#191919C8",
"mini-map-canvas": "#C8C8C819",
"mini-map-canvas-outline": "#C8C8C8C8",
"mini-map-link": "#BB96F6BF",
"mini-map-link-selected": "#BB96F6FF",
"mini-map-node-background": "#C8C8C864",
"mini-map-node-background-hovered": "#C8C8C8FF",
"mini-map-node-background-selected": "#C8C8C8FF",
"mini-map-node-outline": "#C8C8C864",
"mini-map-outline": "#96969664",
"mini-map-outline-hovered": "#969696C8",
"node-background": "#323232FF",
"node-background-hovered": "#4B4B4BFF",
"node-background-selected": "#4B4B4BFF",
"node-outline": "#646464FF",
"pin": "#F5CB25FF",
"pin-hovered": "#FA8335FF",
"title-bar": "#7F61AFFF",
"title-bar-hovered": "#B98EFFFF",
"title-bar-selected": "#CAADF8FF"
},
"implot": {
"axis-bg": "#00000000",
"axis-bg-active": "#00000000",
"axis-bg-hovered": "#00000000",
"axis-grid": "#FFFFFF3F",
"axis-text": "#FFFFFFFF",
"axis-tick": "#00000000",
"crosshairs": "#FFFFFF7F",
"error-bar": "#00000000",
"fill": "#00000000",
"frame-bg": "#FFFFFF11",
"inlay-text": "#FFFFFFFF",
"legend-bg": "#141414EF",
"legend-border": "#6D6D7F7F",
"legend-text": "#FFFFFFFF",
"line": "#00000000",
"marker-fill": "#00000000",
"marker-outline": "#00000000",
"plot-bg": "#0000007F",
"plot-border": "#6D6D7F7F",
"selection": "#FF9900FF",
"title-text": "#FFFFFFFF"
},
"text-editor": {
"background": "#111111FF",
"breakpoint": "#FF200040",
"char-literal": "#E0A070FF",
"comment": "#6473A2FF",
"current-line-edge": "#303030FF",
"current-line-fill": "#00000040",
"current-line-fill-inactive": "#80808040",
"cursor": "#E0E0E0FF",
"default": "#7F7F7FFF",
"doc-comment": "#206020FF",
"error-marker": "#D83545FF",
"doc-global-comment": "#2B9C89FF",
"identifier": "#AFAFAFFF",
"keyword": "#F87AC4FF",
"known-identifier": "#6CF982FF",
"line-number": "#585858FF",
"multi-line-comment": "#206040FF",
"number": "#BD93F9FF",
"preproc-identifier": "#A040C0FF",
"preprocessor": "#202020FF",
"preprocessor-deactivated": "#4F4F4F45",
"punctuation": "#FFFFFFFF",
"selection": "#23313DFF",
"string": "#F1F892FF"
}
},
"styles": {
"imgui": {
"alpha": 1.0,
"button-text-align": [
0.5,
0.5
],
"cell-padding": [
4.0,
2.0
],
"child-border-size": 1.0,
"child-rounding": 0.0,
"disabled-alpha": 0.6000000238418579,
"frame-border-size": 0.0,
"frame-padding": [
4.0,
3.0
],
"frame-rounding": 0.0,
"grab-min-size": 12.0,
"grab-rounding": 0.0,
"indent-spacing": 10.0,
"item-inner-spacing": [
4.0,
4.0
],
"item-spacing": [
8.0,
4.0
],
"popup-border-size": 1.0,
"popup-rounding": 0.0,
"scrollbar-rounding": 9.0,
"scrollbar-size": 14.0,
"selectable-text-align": [
0.0,
0.0
],
"tab-rounding": 4.0,
"window-border-size": 1.0,
"window-min-size": [
32.0,
32.0
],
"window-padding": [
8.0,
8.0
],
"window-rounding": 0.0,
"window-shadow-angle": 0.0,
"window-shadow-offset": 0.0,
"window-shadow-size": 64.0,
"window-title-align": [
0.0,
0.5
]
},
"imhex": {
"popup-alpha": 0.6499999761581421,
"window-blur": 0.0
},
"imnodes": {
"grid-spacing": 24.0,
"link-hover-distance": 10.0,
"link-line-segments-per-length": 0.10000000149011612,
"link-thickness": 3.0,
"mini-map-offset": [
4.0,
4.0
],
"mini-map-padding": [
8.0,
8.0
],
"node-border-thickness": 1.0,
"node-corner-rounding": 4.0,
"node-padding": [
8.0,
8.0
],
"pin-circle-radius": 4.0,
"pin-hover-radius": 10.0,
"pin-line-thickness": 1.0,
"pin-offset": 0.0,
"pin-quad-side-length": 7.0,
"pin-triangle-side-length": 9.5
},
"implot": {
"annotation-padding": [
2.0,
2.0
],
"digital-bit-gap": 4.0,
"digital-bit-height": 8.0,
"error-bar-size": 5.0,
"error-bar-weight": 1.5,
"fill-alpha": 1.0,
"fit-padding": [
0.0,
0.0
],
"label-padding": [
5.0,
5.0
],
"legend-inner-padding": [
5.0,
5.0
],
"legend-padding": [
10.0,
10.0
],
"legend-spacing": [
5.0,
0.0
],
"line-weight": 1.0,
"major-grid-size": [
1.0,
1.0
],
"major-tick-len": [
10.0,
10.0
],
"major-tick-size": [
1.0,
1.0
],
"marker-size": 4.0,
"marker-weight": 1.0,
"minor-alpha": 0.25,
"minor-grid-size": [
1.0,
1.0
],
"minor-tick-len": [
5.0,
5.0
],
"minor-tick-size": [
1.0,
1.0
],
"mouse-pos-padding": [
10.0,
10.0
],
"plot-border-size": 1.0,
"plot-default-size": [
400.0,
300.0
],
"plot-min-size": [
200.0,
150.0
],
"plot-padding": [
10.0,
10.0
]
}
}
}

View File

@@ -143,8 +143,8 @@
"error-marker": "#E782847F",
"unknown-identifier": "#E782847F",
"error-text": "#E782847F",
"debug-text": "#E782847F",
"warning-text": "#E782847F",
"debug-text": "#737994FF",
"warning-text": "#A5ADCEFF",
"pattern-variable": "#949CBBFF",
"function-parameter": "#949CBBFF",
"function-variable": "#949CBBFF",
@@ -160,13 +160,13 @@
"typedef": "#949CBBFF",
"user-defined-type": "#949CBBFF",
"keyword": "#CA9EE6FF",
"builtin-type": "#E78284FF",
"known-identifier": "#E78284FF",
"line-number": "#838BA7FF",
"block-comment": "#737994FF",
"multi-line--comment": "#737994FF",
"number": "#EF9F76FF",
"preprocessor-identifier": "#BABBF1FF",
"directive": "#A5ADCEFF",
"operator": "#C6D0F5FF",
"preproc-identifier": "#BABBF1FF",
"preprocessor": "#A5ADCEFF",
"punctuation": "#C6D0F5FF",
"separator": "#C6D0F5FF",
"selection": "#6268807F",
"string": "#A6D189FF"

View File

@@ -143,8 +143,8 @@
"error-marker": "#D20F397F",
"unknown-identifier": "#D20F397F",
"error-text": "#D20F397F",
"debug-text": "#D20F397F",
"warning-text": "#D20F397F",
"debug-text": "#9CA0B0FF",
"warning-text": "#6C6F85FF",
"pattern-variable": "#7C7F93FF",
"function-parameter": "#7C7F93FF",
"function-variable": "#7C7F93FF",
@@ -160,13 +160,13 @@
"typedef": "#7C7F93FF",
"user-defined-type": "#7C7F93FF",
"keyword": "#8839EFFF",
"builtin-type": "#D20F39FF",
"known=identifier": "#D20F39FF",
"line-number": "#8C8FA1FF",
"block-comment": "#9CA0B0FF",
"multi-line-comment": "#9CA0B0FF",
"number": "#FE640BFF",
"preprocessor-identifier": "#7287FDFF",
"directive": "#6C6F85FF",
"operator": "#4C4F69FF",
"preproc-identifier": "#7287FDFF",
"preprocessor": "#6C6F85FF",
"punctuation": "#4C4F69FF",
"separator": "#4C4F69FF",
"selection": "#ACB0BE7F",
"string": "#40A02BFF"

Some files were not shown because too many files have changed in this diff Show More