From c607910f1cbac2beb4be615cefef93c0922bcf7a Mon Sep 17 00:00:00 2001 From: Jens Steube Date: Thu, 29 May 2025 15:10:09 +0200 Subject: [PATCH] The Assimilation Bridge (Additional Documents) --- ...hashcat-assimilation-bridge-development.md | 133 ++++++++++++++++ docs/hashcat-python-plugin-quickstart.md | 144 ++++++++++++++++++ 2 files changed, 277 insertions(+) create mode 100644 docs/hashcat-assimilation-bridge-development.md create mode 100644 docs/hashcat-python-plugin-quickstart.md diff --git a/docs/hashcat-assimilation-bridge-development.md b/docs/hashcat-assimilation-bridge-development.md new file mode 100644 index 000000000..d82604cd1 --- /dev/null +++ b/docs/hashcat-assimilation-bridge-development.md @@ -0,0 +1,133 @@ + +## Developer Section + +The following section is for plugin and bridge developers. It contains low-level implementation details. It's a first draft, expect more details to fill in soon. + +--- + +## Plugin Integration and Bridge Registration + +Plugins can opt in to bridge support by adding: + +```c +static const u64 BRIDGE_TYPE = BRIDGE_TYPE_MATCH_TUNINGS + | BRIDGE_TYPE_LAUNCH_LOOP; +static const char *BRIDGE_NAME = "scrypt_jane"; +``` + +* `BRIDGE_NAME` tells Hashcat which bridge to load (e.g., `bridge_scrypt_jane.so`). +* `BRIDGE_TYPE` indicates which backend kernel functions the bridge will override: + + * `BRIDGE_TYPE_LAUNCH_LOOP` + * `BRIDGE_TYPE_LAUNCH_LOOP2` + +Hashcat loads the bridge dynamically and uses it for any declared replacements. It's the developer's responsibility to ensure compatibility. That typically means the handling of the `tmps` variable relevant in the `kernel_loop` and how it changes over time. Hashcat will take care of copying the data from and to the compute backend (GPU) buffers. + +--- + +## How Bridges Work + +When Hashcat starts with a plugin that specifies a bridge, it loads the bridge and invokes its initialization function. The bridge must then discover its internal compute units, called *bridge units*. This is done manually by the bridge developer. + +Each bridge unit maps to one virtual backend device, which allows asynchronous and independent parallel execution. + +### Virtual Backend Devices + +* Use `-Y` to define how many virtual backend devices to create. +* Use `-R` to bind these virtual devices to a physical backend host (new in v7). + +This structure supports mixed-performance hardware without bottlenecks. + +--- + +## Writing a Bridge + +### File Layout + +Bridges live in the `src/bridges/` directory and consist of a `.c` file and a `.mk` build rule: + +``` +src/bridges/bridge_scrypt_jane.c +src/bridges/bridge_scrypt_jane.mk +``` + +Example build rule: + +``` +bridges/bridge_scrypt_jane.so: src/bridges/bridge_scrypt_jane.c +``` + +Hashcat will automatically load this shared object based on the plugin's `BRIDGE_NAME`. + +### Required Function Exports + +```c +bridge_ctx->platform_init = platform_init; +bridge_ctx->platform_term = platform_term; +bridge_ctx->get_unit_count = get_unit_count; +bridge_ctx->get_unit_info = get_unit_info; +bridge_ctx->get_workitem_count = get_workitem_count; +bridge_ctx->thread_init = BRIDGE_DEFAULT; +bridge_ctx->thread_term = BRIDGE_DEFAULT; +bridge_ctx->salt_prepare = salt_prepare; +bridge_ctx->salt_destroy = salt_destroy; +bridge_ctx->launch_loop = launch_loop; +bridge_ctx->launch_loop2 = BRIDGE_DEFAULT; +bridge_ctx->st_update_hash = BRIDGE_DEFAULT; +bridge_ctx->st_update_pass = BRIDGE_DEFAULT; +``` + +> Use `BRIDGE_DEFAULT` when no function implementation is required. + +### Mandatory Functions + +The following functions must be defined: + +```c +CHECK_MANDATORY (bridge_ctx->platform_init); +CHECK_MANDATORY (bridge_ctx->platform_term); +CHECK_MANDATORY (bridge_ctx->get_unit_count); +CHECK_MANDATORY (bridge_ctx->get_unit_info); +CHECK_MANDATORY (bridge_ctx->get_workitem_count); +``` + +### Function Roles + +**platform\_init** +Called at startup. Responsible for initialization. This might include loading libraries, connecting to remote endpoints, or setting up hardware APIs. Returns a context pointer. + +**platform\_term** +Final cleanup logic. Frees any context data allocated during initialization. + +**get\_unit\_count** +Returns the number of available units. For example, return `2` if two FPGAs are detected. + +**get\_unit\_info** +Returns a human-readable description of a unit, like "Python v3.13.3". + +**get\_workitem\_count** +Returns the number of password candidates to process per invocation. + +**thread\_init** +Optional. Use for per-thread setup, such as creating a new Python interpreter. + +**thread\_term** +Optional. Use for per-thread cleanup. + +**salt\_prepare** +Called once per salt. Useful for preprocessing or storing large salt/esalt buffers. + +**salt\_destroy** +Optional cleanup routine for any salt-specific memory. + +**launch\_loop** +Main compute function. Replaces the traditional `_loop` kernel. + +**launch\_loop2** +Secondary compute function. Replaces `_loop2` if needed. + +**st\_update\_hash** +Optionally override the module's default self-test hash. + +**st\_update\_pass** +Optionally override the module's default self-test password. diff --git a/docs/hashcat-python-plugin-quickstart.md b/docs/hashcat-python-plugin-quickstart.md new file mode 100644 index 000000000..1c27617aa --- /dev/null +++ b/docs/hashcat-python-plugin-quickstart.md @@ -0,0 +1,144 @@ + +# Hashcat Python Plugin Developer Guide + +## Introduction + +This guide walks you through building custom hash modes in **pure Python** using Hashcat v7's Python plugin from the new assimilation bridge. Whether you're experimenting with a new algorithm, supporting a proprietary format, hacking a prototype, or just writing hashing logic in a high-level language, this plugin interface makes it fast and easy. + +No C required. No recompilation. Just write your logic in pure python code in `calc_hash()`. + +You can use any python modules you want. + +--- + +## Quick Start + +Hashcat mode `73000` is preconfigured to load a generic Python plugin source file from `Python/generic_hash_mp.py`: + +``` +hashcat -m 73000 -b +``` + +You can edit the `Python/generic_hash_mp.py`, or override the plugin source file with: + +``` +hashcat -m 73000 --bridge-parameter1=my_hash.py hash.txt wordlist.txt ... +``` + +--- + +## Yescrypt in One Line + +### Generate a Yescrypt Test Hash + +``` +echo password | mkpasswd -s -m yescrypt +``` + +Example output: + +``` +$y$j9T$uxVFACnNnGBakt9MLrpFf0$SmbSZAge5oa1BfHPBxYGq3mITgHeO/iG2Mdfgo93UN0 +``` + +### Prepare Hash Line for Hashcat + +``` +$y$j9T$uxVFACnNnGBakt9MLrpFf0$SmbSZAge5oa1BfHPBxYGq3mITgHeO/iG2Mdfgo93UN0*$y$j9T$uxVFACnNnGBakt9MLrpFf0$ +``` + +(Use the full hash before the `*`, and the salt portion after the `*`.) + +### Plugin Code + +Install the module: + +``` +pip install pyescrypt +``` + +Then in your plugin (either `generic_hash_mp.py` for -m 73000 or `generic_hash_sp.py` for -m 72000) + +```python +from pyescrypt import Yescrypt,Mode + +# Self-Test pair +ST_HASH = "$y$j9T$uxVFACnNnGBakt9MLrpFf0$SmbSZAge5oa1BfHPBxYGq3mITgHeO/iG2Mdfgo93UN0*$y$j9T$uxVFACnNnGBakt9MLrpFf0$" +ST_PASS = "password" + +def calc_hash(password: bytes, salt: dict) -> str: + return Yescrypt(n=4096, r=32, p=1, mode=Mode.MCF).digest(password=password, settings=hcshared.get_salt_buf(salt)).decode('utf8') +``` + +That's it - full Yescrypt support in Hashcat with a single line of code. + +### Run + +``` +hashcat -m 73000 yescrypt.hash wordlist.txt +``` + +--- + +## Debugging Without Hashcat + +You can run your plugin as a standalone script: + +``` +python3 generic_hash.py +``` + +It reads passwords from stdin and prints the result of `calc_hash()`: + +``` +echo "password" | python3 generic_hash_mp.py +``` + +Note that you probably want to inline the correct salt value, see the `main` section in the code. + +--- + +## Windows and Linux/macOS + +There are significant differences between Windows and Linux/macOS when embedding Python as done here. It's a complex issue, and we hope future Python developments toward a fully GIL-free mode will resolve it. For now, we must work around platform-specific behavior. + +### On Windows + +The `multiprocessing` module is not fully supported in this embedded setup. As a result, only one thread can run effectively. While the `threading` module does work, most cryptographic functions like `sha256()` block the GIL. CPU-intensive algorithms such as 10,000 iterations of `sha256()` will monopolize the GIL, making the program effectively single-threaded. + +### On Linux/macOS + +The `multiprocessing` module works correctly, enabling full CPU utilization through parallel worker processes. + +### Free-threaded Python (3.13+) + +Python 3.13 introduces optional GIL-free support. This allows multithreading to work even in embedded Python, both on Linux and Windows. Hashcat leverages this through two modes: + +- `-m 72000`: Uses free-threaded Python (no multiprocessing) +- `-m 73000`: Uses GIL-bound Python with multiprocessing + +Multiprocessing (73000) supports most modules and is generally better for real-world workloads, but it works only on Linux. Developers may use `-m 73000` on Linux for performance and `-m 72000` on Windows for development, provided their code does not rely on modules that require `cffi` because this as of now lacks support for running with Python free-treaded ABI. + +--- + +## Python 3.13 Requirement + +The `-m 72000` mode requires Python 3.13 due to its reliance on the new free-threading feature. This feature is not available in earlier versions. + +### Why Python 3.13 Isn't Preinstalled + +Several Linux distributions, including Ubuntu 24.04, do not ship with Python 3.13 because it was released after the distro’s feature freeze. You will likely need to install it manually. + +### Installing Python 3.13 + +**On Windows**: Use the official installer and ensure you check the "Install free-threaded" option - it's disabled by default. + +**On Linux/macOS**: Use `pyenv`. It's the easiest way to install and manage Python versions: + +``` +pyenv install 3.13t +pyenv local 3.13t +``` + +This makes it easy to manage `pip` packages without global installs or virtual environments. +