hashcat/docs/changes.txt

* changes v3.10 -> v3.20:

The hashcat core was completely refactored to be a MT-safe library (libhashcat).
The goal was to help developers include hashcat into distributed clients or GUI frontends.
The CLI (hashcat.bin or hashcat.exe) works as before but from a technical perspective it's a library frontend.

##
## Features
##

- New option --speed-only: Quickly provides cracking speed per device based on the user hashes and selected options, then quit
- New option --keep-guessing: Continue cracking hashes even after they have been cracked (to find collisions)
- New option --restore-file-path: Manually override the path to the restore file (useful if we want all session files in the same folder)
- New option --opencl-info: Show details about OpenCL compatible devices like an embedded clinfo tool (useful for bug reports)
- Documents: Added colors for warnings (yellow) and errors (red) instead of WARNING: and ERROR: prefix
- Documents: Added hints presented to the user about optimizing performance while hashcat is running
- Hardware management: Support --gpu-temp-retain for AMDGPU-Pro driver
- Hardware management: Support --powertune-enable for AMDGPU-Pro driver
- Password candidates: Allow words of length > 31 in wordlists for -a 0 for some slow hashes if no rules are in use
- Password candidates: Do not use $HEX[] if the password candidate is a valid UTF-8 string and print out as-is
- Pause mode: Allow quit program also if in pause mode
- Pause mode: Ignore runtime limit in pause mode
- Status view: Show core-clock, memory-clock and execution time in benchmark-mode in case --machine-readable is activated
- Status view: Show temperature, coreclock, memoryclock, fanspeed and pci-lanes for devices using AMDGPU-Pro driver
- Status view: Show the current first and last password candidate test queued for execution per device (as in JtR)
- Status view: Show the current position in the queue for both base and modifier (Example: Wordlist 2/5)
- Markov statistics: Update hashcat.hcstat which is used as reference whenever the user defines a mask
- Charsets: Added lowercase ascii hex (?h) and uppercase ascii hex (?H) as predefined charsets

##
## Algorithms
##

- Added hash-mode 14000 = DES (PT = $salt, key = $pass)
- Added hash-mode 14100 = 3DES (PT = $salt, key = $pass)
- Added hash-mode 14400 = SHA1(CX)
- Added hash-mode 99999 = Plaintext
- Extended hash-mode 3200 = bcrypt: Accept signature $2b$ (February 2014)
- Improved hash-mode 8300 = DNSSEC: Additional parsing error detection

##
## Bugs
##

- Custom charset from file parsing code did not return an error if an error occured
- Fix some clSetKernelArg() size error that caused slow modes to not work anymore in -a 1 mode
- Hash-mode 11600 = (7-Zip): Depending on input hash a clEnqueueReadBuffer(): CL_INVALID_VALUE error occured
- Hash-mode 22 = Juniper Netscreen/SSG (ScreenOS): Fix salt length for -m 22 in benchmark mode
- Hash-Mode 5500 = NetNTLMv1 + ESS: Fix loading of NetNTLMv1 + SSP hash
- Hash-mode 6000 = RipeMD160: Fix typo in array index number
- If cracking a hash-mode using unicode passwords, length check of a mask was not taking into account
- If cracking a large salted hashlist the wordlist reject code was too slow to handle it, leading to 0H/s
- Null-pointer dereference in outfile-check shutdown code when using --outfile-check-dir, leading to segfault
- On startup hashcat tried to access the folder defined in INSTALL_FOLDER, leading to segfault if that folder was not existing
- Random rules generator code used invalid parameter for memory copy function (M), leading to use of invalid rule
- Sanity check for --outfile-format was broken if used in combination with --show or --left

##
## Workarounds
##

- Workaround added for AMDGPU-Pro OpenCL runtime: Failed to compile hash-mode 10700 = PDF 1.7 Level 8
- Workaround added for AMDGPU-Pro OpenCL runtime: Failed to compile hash-mode 1800 = sha512crypt
- Workaround added for NVidia OpenCL runtime: Failed to compile hash-mode 6400 = AIX {ssha256}
- Workaround added for NVidia OpenCL runtime: Failed to compile hash-mode 6800 = Lastpass + Lastpass sniffed
- Workaround added for OSX OpenCL runtime: Failed to compile hash-mode 10420 = PDF 1.1 - 1.3 (Acrobat 2 - 4)
- Workaround added for OSX OpenCL runtime: Failed to compile hash-mode 1100 = Domain Cached Credentials (DCC), MS Cache
- Workaround added for OSX OpenCL runtime: Failed to compile hash-mode 13800 = Windows 8+ phone PIN/Password
- Workaround added for pocl OpenCL runtime: Failed to compile hash-mode 5800 = Android PIN

##
## Performance
##

- Improved performance for rule-based attacks for _very_ fast hashes like MD5 and NTLM by 30% or higher
- Improved performance for cracking DEScrypt on AMD, from 373MH/s to 525MH/s
- Improved performance for cracking raw DES-based algorithms (like LM) on AMD, from 1.6GH/s to 12.5GH/s
- Improved performance for cracking raw SHA256-based algorithms using meet-in-the-middle optimization, reduces 7/64 steps
- Improved performance by reducing maximum number of allowed function calls per rule from 255 to 31
- Improved performance by update the selection when to use #pragma unroll depending on OpenCL runtime vendor

- Full performance comparison sheet v3.10 vs. v3.20: https://docs.google.com/spreadsheets/d/1B1S_t1Z0KsqByH3pNkYUM-RCFMu860nlfSsYEqOoqco/edit#gid=1591672380

##
## Technical
##

- Autotune: Do not run any caching rounds in autotune in DEBUG mode if -n and -u are specified
- Bash completion: Removed some v2.01 leftovers in the bash completion configuration
- Benchmark: Do not control fan speed in benchmark mode
- Benchmark: On OSX, some hash-modes can't compile because of OSX OpenCL runtime. Skip them and move on to the next
- Building: Added Makefile target "main_shared", a small how-to-use libhashcat example
- Building: Added many additional compiler warning flags in Makefile to improve static code error detection
- Building: Added missing includes for FreeBSD
- Building: Added some types for windows only in case _BASETSD_H was not set
- Building: Changed Makefile to strip symbols in the linker instead of the compiler
- Building: Defined NOMINMAX macro to prevent definition min and max macros in stdlib header files
- Building: Enabled ASLR and DEP for Windows builds
- Building: Fixed almost all errors reported by cppcheck and scan-build
- Building: On OSX, move '-framework OpenCL' from CFLAGS to LDFLAGS
- Building: On OSX, use clang as default compiler
- Building: Support building on Msys2 environment
- Building: Use .gitmodules to simplify the OpenCL header dependency handling process
- Charsets: Added DES_full.charset
- Data Types: Replaced all integer macros with enumerator types
- Data Types: Replaced all integer variables with true bool variables in case they are used as a bool
- Data Types: Replaced all string macros with static const char types
- Data Types: Replaced all uint and uint32_t to u32
- Data Types: Replaced atoi() with atoll(). Eliminates sign conversion warnings
- Documents: Added docs/credits.txt
- Documents: Added docs/team.txt
- Documents: Changed rules.txt to match v3.20 limitations
- Error handling (file handling): Fixed a couple of filepointer leaks
- Error handling (format strings): Fixed a few printf() formats, ex: use %u instead of %d for uint32_t
- Error handling (memory allocation): Removed memory allocation checks, just print to stderr instead
- Error handling (startup): Added some missing returncode checks to get_exec_path()
- Fanspeed: Check both fanpolicy and fanspeed returncode and disable retain support if any of them fail
- Fanspeed: Minimum fanspeed for retain support increased to 33%, same as NV uses as default on windows
- Fanspeed: Reset PID controler settings to what they were initially
- Fanspeed: Set fan speed to default on quit
- File handling: Do a single write test (for files to be written later) directly on startup
- File locking: Use same locking mechanism in potfile as in outfile
- Hardware management: Fixed calling conventions for ADL, NvAPI and NVML on windows
- Hardware management: Improved checking for successfull load of the NVML API
- Hardware management: In case fanspeed can not be set, disable --gpu-temp-retain automatically
- Hardware management: In case of initialization error show it only once to the user on startup
- Hardware management: Refactored all code to return returncode (0 or -1) instead of data for more easy error handling
- Hardware management: Refactored macros to real functions
- Hardware management: Removed kernel exec timeout detection on NVIDIA, should no longer occur due to autotune
- Hardware management: Replaced NVML registry functions macros with their ascii versions (Adds NVML support for XP)
- Hashlist loading: Do not load data from hashfile if hashfile changed during runtime
- Kernel cache: Fixed checksum building on oversized device version or driver version strings
- Logging: Improved variable names in hashcat.log
- Loopback: Refactored --loopback support completely, no longer a recursive function
- Memory management: Fixed some memory leaks on shutdown
- Memory management: Got rid of all global variables
- Memory management: Got rid of local_free() and global_free(), no longer required
- Memory management: Refactored all variables with HCBUFSIZ_LARGE size from stack to heap, OSX doesn't like that
- OpenCL Headers: Select OpenCL headers tagged for OpenCL 1.2, since we use -cl-std=CL1.2
- OpenCL Kernels: Added const qualifier to variable declaration of matching global memory objects
- OpenCL Kernels: Got rid of one global kernel_threads variable
- OpenCL Kernels: Moved OpenCL requirement from v1.1 to v1.2
- OpenCL Kernels: Recognize reqd_work_group_size() values from OpenCL kernels and use them in the host if possible
- OpenCL Kernels: Refactored common function append_0x01()
- OpenCL Kernels: Refactored common function append_0x02()
- OpenCL Kernels: Refactored common function append_0x80()
- OpenCL Kernels: Refactored rule function append_block1()
- OpenCL Kernels: Refactored rule function rule_op_mangle_delete_last()
- OpenCL Kernels: Refactored rule function rule_op_mangle_dupechar_last()
- OpenCL Kernels: Refactored rule function rule_op_mangle_rotate_left()
- OpenCL Kernels: Refactored rule function rule_op_mangle_rotate_right()
- OpenCL Kernels: Support mixed kernel thread count for mixed kernels in the same source file
- OpenCL Kernels: Switch from clz() to ffz() for bitsliced algorithms
- OpenCL Kernels: Using platform vendor name is better than using device vendor name for function detection
- OpenCL Runtime: Updated AMDGPU-Pro and AMD Radeon driver version check
- OpenCL Runtime: Updated Intel OpenCL runtime version check
- OpenCL Runtime: Updated NVIDIA driver version check
- Password candidates: The maximum word length in a wordlist is 31 not 32, because 0x80 will eventually be appended
- Potfile: Base logic switched; Assuming the potfile is larger than the hashlist it's better to load hashlist instead of potfile entries
- Potfile: In case all hashes were cracking using potfile abort and inform user
- Restore: Automatically unlink restore file if all hashes have been cracked
- Restore: Do not unlink restore file if restore is disabled
- Rules: Refactored macros to real functions
- Status: Added Input.Queue.Base and Input.Queue.Mod to help the user better understand this concept
- Status: Do not wait for the progress mutex to read and store speed timer
- Status: Do not show Recovered/Time when cracking < 1000 hashes
- Status: Do not show Recovered/Time as floats but as integers to reduce over-information
- Tests: Removed rules_test/ subproject: Would require total rewrite but not used in a long time
- Threads: Replaced all calls to getpwuid() with getpwuid_r() to ensure thread safety
- Threads: Replaced all calls to gmtime() with gmtime_r() to ensure thread safety
- Threads: Replaced all calls to strtok() with strtok_r() to ensure thread safety
- Wordlists: Use larger counter variable to handle larger wordlists (that is > 2^32 words)
- X11: Detect missing coolbits and added some help text for the user how to fix it

* changes v3.00 -> v3.10:

##
## Improvements
##

- Added mask display to modes 3, 6, and 7. Allows the user to see the custom character set used during the run
- Make Linux build POSIX compatible; Also allow it to actually compile on musl-libc systems
- Add support to compile on FreeBSD
- Make use of cl_context_properties[] to clCreateContext(), even if OpenCL specification allow the use of NULL, some runtimes fail without
- The Time.Estimated attribute in status display should also show --runtime limit if user set it
- Fix some strict aliasing rule violation on older compilers
- Fix some variable initializers on older compilers
- Replace DARWIN macro with compiler predefined macro __APPLE__
- Replace LINUX macro with compiler predefined macro __linux__
- Allow the use of enc_id == 0 in hash-mode 10600 and 10700 as it takes no part in the actual computation
- Get rid of exit() calls in OpenCL wrapper library with the goal to have a better control which error can be ignored under special circumstances
- Do not error and exit if an OpenCL platform has no devices, just print a warning and continue with the next platform
- Workaround for OpenCL runtimes which do not accept -I parameter in the OpenCL kernel build options even if this is an OpenCL standard option
- Workaround for OpenCL runtimes which do accept -I parameter in the OpenCL kernel build options, but do not allow quotes
- Output cracked hashes on Windows using \r\n and not \n
- Replace RegGetValue() with RegQueryValueEx() to enable Windows XP 32 bit compatibility
- Slightly increased NVidias rule-processing performance by using generic instructions instead of byte_perm()
- Add support for @ rule (RULE_OP_MANGLE_PURGECHAR) to use on GPU
- Add support for --outfile (short -o) to be used together with --stdout
- Skip periodic status output whenever --stdout is used together with stdin mode, but no outfile was specified
- Show error message if --show is used together with --outfile-autohex-disable (this is currently not supported)
- Show error message if --skip/--limit is used together with mask files or --increment
- Workaround for NVidia OpenCL runtime bug causing -m 6223 to not crack any hashes even with the correct password candidate

##
## Bugs
##

- Fixed a bug where CRAM MD5 checked salt length instead of hash length
- Fixed a bug where hashcat is suppressing --machine-readable output in the final status update
- Fixed a bug where hashcat did not check the return of realpath() and crashes uncontrolled if the path does not exist
- Fixed a bug where hashcat crashes for accessing deallocated buffer if user spams "s" shortly before hashcat shuts down
- Fixed a bug where hashcat crashes in case of a scrypt P setting > 1
- Fixed a bug where hashcat did not correctly use the newly cracked plains whenever --loopback or the induction folder was used
- Fixed a bug where hashcat did not correctly remove hashes of type WPA/WPA2 even if present in potfile
- Fixed a bug where hashcat reported an invalid password for a zero-length password in LM
- Fixed a bug where hashcat did not take into account how long it takes to prepare a session when auto-aborting with --runtime is in use
- Fixed a bug where some kernels used COMPARE_M_SIMD instead of COMPARE_S_SIMD in singlehash mode

##
## Algorithms
##

- Added new hash-mode 13900 = OpenCart

* changes v2.01 -> v3.00:

This release markes the fusion of "hashcat" and "oclHashcat" into "hashcat".
It combines all features of all hashcat projects in one project.

##
## Features
##

- Support for Apple OpenCL runtime
- Support for NVidia OpenCL runtime (replaces CUDA)
- Support for Mesa (Gallium) OpenCL runtime
- Support for pocl OpenCL runtime
- Support for Khronos' OSS OpenCL reference implementation for building
- Support to utilize OpenCL devices-types other than GPU, ex: CPU and FPGA
- Support to utilize multiple different OpenCL platforms in parallel, ex: AMD + NV
- Support to utilize multiple different OpenCL device-types in parallel, ex: GPU + CPU
- Added option --opencl-platform to select a specific OpenCL platform
- Added option --opencl-device-types select specific OpenCL device types
- Added option --opencl-vector-width to override automatically selected vector-width size
- Added makefile native compilation target
- Added makefile install and uninstall targets
- Added autotuning engine and user-configurable tuning database
- Added current engine clock, current memory clock and pci-e lanes to the status display
- Added support for --gpu-temp-retain for NVidia GPU, both Linux and Windows
- Added execution timer of the running kernel to the status display
- Added command prompt to quit at next restore checkpoint
- Added human-readable error message for the OpenCL error codes
- Added option --potfile-path to override potfile path
- Added option --veracrypt-keyfile to set Keyfiles used, can be multiple
- Added option --veracrypt-pim to set the VeraCrypt personal iterations multiplier
- Added option --machine-readable for easier parsing of output
- Added option --powertune-enable to work with NVidia devices as well, not just AMD
- Added option --stdout to print candidates instead of trying to crack a hash

##
## Algorithms
##

- Added new hash-mode   125 = ArubaOS
- Added new hash-mode 12900 = Android FDE (Samsung DEK)
- Added new hash-mode 13000 = RAR5
- Added new hash-mode 13100 = Kerberos 5 TGS-REP etype 23
- Added new hash-mode 13200 = AxCrypt
- Added new hash-mode 13300 = AxCrypt in memory SHA1
- Added new hash-mode 13400 = Keepass 1 (AES/Twofish) and Keepass 2 (AES)
- Added new hash-mode 13500 = PeopleSoft PS_TOKEN
- Added new hash-mode 13600 = WinZip
- Added new hash-mode 137** = VeraCrypt
- Added new hash-mode 13800 = Windows 8+ phone PIN/Password

##
## Performance
##

- Full Table: https://docs.google.com/spreadsheets/d/1B1S_t1Z0KsqByH3pNkYUM-RCFMu860nlfSsYEqOoqco/edit#gid=0

##
## Improvements
##

- Reordering of files to help integration into linux distributions ~/.hashcat etc
- Use a profile directory to write temporary files (session, potfile etc.)
- Workaround dependencies on AMD APP-SDK AMD ADL, NV CUDA-SDK, NV ForceWare, NVML and NVAPI; they are no longer required
- Load external libraries dynamic at runtime instead of link them static at compile-time
- Benchmark accuracy improved; Is now on par to: singlehash -a 3 -w 3 ?b?b?b?b?b?b?b
- Benchmark no longer depends on a fixed time
- Removed option --benchmark-mode, therefore support --workload-profile in benchmark-mode
- Enabled support of --machine-readable in combination with --benchmark for automated benchmark processing
- Replaced --status-automat entirely with --machine-readable to make it more consistent among benchmark and non-benchmark mode
- Extended support from 14 to 255 functions calls per rule
- Extended password length up to 32 for 7zip
- Extended salt length up to 55 for raw hash types, eg: md5($pass.$salt)
- Extended version information
- Removed some duplicate rules in T0XlCv1, d3ad0ne and dive
- Redesigned changes.txt layout
- Redesigned --help menu layout

##
## Bugs
##

- Fixed a bug in speed display: In some situation, especially with slow hashes or lots of salts, it showed a speed of 0H/s
- Fixed a bug in restore handling: user immediately aborting after restart broke the restore file
- Fixed a bug in line counter: conditional jump or move depends on an uninitialised value
- Fixed a bug in rule-engine for NVidia devices: code for left- and right-shift were switched
- Fixed a bug in dive.rule: rules were not updated after the function 'x' was renamed to 'O'
- Fixed a bug in memory allocation "OpenCL -4 error": used unitialized value in a special situation
- Fixed a bug in memory handling: heap buffer overflow
- Fixed a bug in memory handling: out of bounds access
- Fixed a bug in implementation of DCC2: forced default iteration count for hashes to 10240
- Fixed a bug in implementation of WPA/WPA2: MAC and nonce stay one their original position as in the hccap file
- Fixed a bug in implementation of GOST R 34.11-94: zero length passwords were not cracked

##
## Technical
##

- Removed deprecated GCC version check requirement
- Removed NPROCS from Makefile, let make automatically detect the optimal number of parallel threads
- Dropped all C++ overloading functions to normal function which helps support more OpenCL platforms
- Renamed functions in common.h to emphasize their purpose
- Refactorized fast-hash kernels to enable SIMD on all OpenCL platforms
- Refactorized SIMD handling: SIMD the inner-loop not the outer-loop to save registers
- Workaround missing clEnqueueFillBuffer() support in certain OpenCL runtimes
- Added amd_bytealign() support in non-AMD OpenCL runtimes
- Added amd_bfe() support in non-AMD OpenCL runtimes
- Added several macros to allow writing optimized code for the different OpenCL platforms
- Replaced typedef for bool with stdbool.h
- Added special DEBUG environment variables to the makefile
- Hashcat now acquires an exclusive lock before writing to any file
- Changed buffers to not use same buffer for both input and output at the same time with snprintf()
- Check for allocatable device-memory depending on kernel_accel amplifier before trying to allocate
- Added additional check for max. ESSID length to prevent possible crashes
- Use a GCC equivalent for __stdcall where applicable
- Synchronize maximum output line size with input line size
- Increased maximum hash line size to 0x50000
- Run weak-hash checks only in straight-attack mode, this greatly reduces code complexity
- Restrict loopback option to straight attack-mode
- Moved rules_optimize to hashcat-utils
- Stick to older libOpenCL in binary package to avoid errors like this: version `OPENCL_2.0' not found
- Tightened hash parser for several algorithms
- Updated old RC4 code in Kerberos 5
- Limited the salt length of Juniper Netscreen/SSG (ScreenOS) hashes to 10
- Updated algorithm used to automatically select an ideal --scrypt-tmto value
- Renamed option --gpu-accel to --kernel-accel
- Renamed option --gpu-loops to --kernel-loops
- Renamed option --gpu-devices to --opencl-devices
- Added inline declaration to functions from simd.c, common.c, rp.c and types_ocl.c to increase performance
- Dropped static declaration from functions in all kernel to achieve OpenCL 1.1 compatibility
- Added -cl-std=CL1.1 to all kernel build options
- Created environment variable to inform NVidia OpenCL runtime to not create its own kernel cache
- Created environment variable to inform pocl OpenCL runtime to not create its own kernel cache
- Dropped special 64-bit rotate() handling for NV, it seems that they've added it to their OpenCL runtime
- Completely get rid of HAVE_ADL, HAVE_NVML and HAVE_NVAPI in sources
- Replaced NVAPI with NVML on windows