hashcat/docs/changes.txt

* changes v3.6.0 -> xxx:

##
## Features
##

- Added support to crack passwords and salts up to length 256
- Added option --optimized-kernel-enable to use faster kernels but limit the maximum supported password- and salt-length
- Added self-test functionality to detect broken OpenCL runtimes on startup
- Added option --self-test-disable to disable self-test functionality on startup
- Added option --wordlist-autohex-disable to disable the automatical conversion of $HEX[] words from the word list
- Added option --example-hashes to show an example hash for each hash-mode

##
## Algorithms
##

- Added hash-mode  2501 = WPA/WPA2 PMK

##
## Bugs
##

- Fixed a memory problem that occured when the OpenCL folder was not found and e.g. the shared and session folder were the same
- Fixed a missing barrier() call in the RACF OpenCL kernel
- Fixed a missing salt length value in benchmark mode for SIP
- Fixed an invalid progress value in status view if words from the base wordlist get rejected because of length
- Fixed an invalid optimization code in kernel 7700 depending on the input hash, causing the kernel to loop forever
- Fixed an integer overflow in innerloop_step and innerloop_cnt variables
- Fixed an integer overflow in masks not skipped when loaded from file
- Fixed an integer overflow in hash buffer size calculation
- Fixed a parser error for mode -m 9820 = MS Office <= 2003 $3, SHA1 + RC4, collider #2
- Fixed a parser error in multiple modes not checking for return code, resulting in negative memory index writes
- Fixed a problem with changed current working directory, for instance by using --restore together with --remove
- Fixed a problem with the conversion to the $HEX[] format: convert/hexify also all passwords of the format $HEX[]
- Fixed the calculation of device_name_chksum; should be done for each iteration
- Fixed the estimated time value whenever the value is very large and overflows
- Fixed the parsing of command line options. It doesn't show two times the same error about an invalid option anymore
- Fixed the parsing of DCC2 hashes by allowing the "#" character within the user name
- Fixed the parsing of descrypt hashes if the hashes do have non-standard characters within the salt
- Fixed the output of --show when used together with the collider modes -m 9710, 9810 or 10410
- Fixed the use of --veracrypt-pim option. It was completely ignored without showing an error
- Fixed the version number used in the restore file header
- Fixed overflow in mangle_dupechar_last function

##
## Improvements
##

- Charset: Add additional DES charsets with corrected parity
- OpenCL Buffers: Do not allocate memory for amplifiers for fast hashes, it's simply not needed
- OpenCL Kernels: Improved rule engine performance by 6% on for NVidia
- OpenCL Kernels: Improved performance of SHA-3 Kernel (keccak) by hardcoding the 0x80 stopbit
- OpenCL Kernels: Move from ld.global.v4.u32 to ld.const.v4.u32 in _a3 kernels
- OpenCL Kernels: Replace bitwise swaps with rotate() versions for AMD
- OpenCL Kernels: Rewritten SIP kernel from scratch
- OpenCL Kernels: Rewritten Keccak kernel to run fully on registers and partially reversed last round
- OpenCL Kernels: Updated default scrypt TMTO to be ideal for latest NVidia and AMD top models
- OpenCL Kernels: Vectorized tons of slow kernels to improve CPU cracking speed
- OpenCL Runtime: Updated AMD ROCm driver version check, warn if version < 1.1
- Startup: Show some attack-specific optimizer constraints on start, eg: minimum and maximum support password- and salt-length
- WPA cracking: Improved nonce-error-corrections mode to use a both positive and negative corrections

##
## Technical
##

- General: Update C standard from c99 to gnu99
- Hash Parser: Improved salt-length checks for generic hash modes
- HCstat File: Add code to read LZMA compressed hashcat.hcstat2
- HCstat File: Add hcstat2 support to enable masks of length up to 256, also adds a filetype header
- OpenCL Kernels: Added code generator for most of the switch_* functions and replaced existing code
- OpenCL Kernels: Declared all include functions as static to reduce binary kernel cache size
- OpenCL Kernels: On AMD GPU, optimized kernels for use with AMD ROCm driver
- OpenCL Kernels: Removed some include functions that are no longer needed to reduce compile time
- OpenCL Runtime: Fall back to 64 threads default (from 256) on AMD GPU to prevent creating too many workitems
- OpenCL Runtime: Forcing OpenCL 1.2 no longer needed. Option removed from build options
- Restore: Fixed the version number used in the restore file header
- Time: added new type for time measurements hc_time_t and related functions to force the use of 64 bit times

* changes v3.5.0 -> v3.6.0:

##
## Algorithms
##

- Added hash-mode   600 = BLAKE2-512
- Added hash-mode 15200 = Blockchain, My Wallet, V2
- Added hash-mode 15300 = DPAPI masterkey file v1 and v2
- Added hash-mode 15400 = ChaCha20
- Added hash-mode 15500 = JKS Java Key Store Private Keys (SHA1)
- Added hash-mode 15600 = Ethereum Wallet, PBKDF2-HMAC-SHA256
- Added hash-mode 15700 = Ethereum Wallet, PBKDF2-SCRYPT

##
## Features
##

- 7-Zip cracking: increased max. data length to 320k and removed AES padding attack to avoid false negatives
- Dictionary cache: Show time spent on dictionary cache building at startup
- Rules: Support added for position 'p' (Nth instance of a character) in host mode (using -j or -k)
- Rules: Support added for rejection rule '_N' (reject plains of length not equal to N) in host mode
- Rules: Support added for rule 'eX'
- Wordlist encoding: Added parameters --encoding-from and --encoding-to to configure wordlist encoding handling
- Wordlist encoding: Support added for internal conversion between user-defined encodings during runtime

##
## Workarounds
##

- Workaround added for NVIDIA NVML library: If libnvidia-ml.so couldn't be loaded, try again using libnvidia-ml.so.1

##
## Improvements
##

- WPA cracking: Improved nonce-error-corrections mode to fix corrupt nonces generated on big-endian devices

##
## Bugs
##

- Fixed a condition that caused hybrid attacks using a maskfile to not select all wordlists from a wordlist folder
- Fixed a memory leak that was present when a user periodically prints hashcat status (using --status-timer)
- Fixed a missing type specifier in a function declaration of the RACF kernel

##
## Technical
##

- Building: In the binary release packages, link libiconv static for Windows binaries
- Dictstat: Structure for dictstat file changed to include --encoding-from and --encoding-to parameters
- OpenCL Runtime: Updated AMDGPU-PRO driver version check, warn if version 17.10 (known to be broken) is detected
- WPA cracking: Reduced --nonce-error-corrections default from 16 to 8 to compensate for speed drop caused by big-endian fixes

* changes v3.40 -> v3.5.0:

##
## Features
##

- WPA cracking: Added support for WPA/WPA2 handshake AP nonce automatic error correction
- WPA cracking: Added parameter --nonce-error-corrections to configure range of error correction

##
## Algorithms
##

- Added hash-mode 15100 = Juniper/NetBSD sha1crypt

##
## Improvements
##

- Abbreviate long hashes to display the Hash.Target status line within 80 characters
- Refactored internal use of esalt to sync with the number of digests instead of the number of salts
- Refactored other output to display within 80 characters without wrapping

##
## Bugs
##

- Fixed a hash validation error when trying to load Android FDE < 4.3 hashes
- Fixed a problem where --keyspace combined with custom charsets incorrectly displayed an error message
- Fixed a problem where --stdout combined with custom charsets incorrectly displayed an error message
- Fixed a problem with parsing and displaying -m 7000 = Fortigate (FortiOS) hashes
- Fixed a race condition after sessions finish, where the input-base was freed but accessed afterwards
- Fixed a typo that resulted in the minimum password length not being correctly initialized
- Fixed --outfile-format formats 11 through 15 to show the correct crack position
- Fixed --remove to apply even when all hashes are either found in the potfile or detected in weak-hash checks

##
## Technical
##

- Building: Added missing prototypes for atlassian_parse_hash function
- Dictionary Cache: Split long status line into multiple lines to stay < 80 chars
- Files: Detect and error when users try to use -r with a parameter which is not a file
- HCCAPX Parser: Added support for a special bit (bit 8) of the message_pair that indicates if replay counters match
- Parameter: Detect and error when users try to use an empty string (length 0) for parameters like --session=
- Parameter: Detect and error when users try to use non-digit input when only digits are expected
- Sessions: Improved string comparison in case user sets --session to "hashcat"
- Status View: Add rejected counter to machine-readable output
- Status View: Rename labels Input.Mode, Input.Base, ... to Guess.Mode, Guess.Base, ...
- Status View: Added a visual indicator to the status screen when checkpoint quit has been requested
- Versions: Changed version naming convention from x.yz to x.y.z

* changes v3.30 -> v3.40:

##
## Features
##

- Added support for loading hccapx files
- Added support for filtering hccapx message pairs using --hccapx-message-pair
- Added support for parsing 7-Zip hashes with LZMA/LZMA2 compression indicator set to a non-zero value
- Added support for decompressing LZMA1/LZMA2 data for -m 11600 = 7-Zip to validate the CRC
- Added support for automatic merge of LM halfes in case --show and --left is used
- Added support for showing all user names with --show and --left if --username was specified
- Added support for GPU temperature management on cygwin build

##
## Algorithms
##

- Added hash-mode  1411 = SSHA-256(Base64), LDAP {SSHA256}
- Added hash-mode  3910 = md5(md5($pass).md5($salt))
- Added hash-mode  4010 = md5($salt.md5($salt.$pass))
- Added hash-mode  4110 = md5($salt.md5($pass.$salt))
- Added hash-mode  4520 = sha1($salt.sha1($pass))
- Added hash-mode  4522 = PunBB
- Added hash-mode  7000 = Fortigate (FortiOS)
- Added hash-mode 12001 = Atlassian (PBKDF2-HMAC-SHA1)
- Added hash-mode 14600 = LUKS
- Added hash-mode 14700 = iTunes Backup < 10.0
- Added hash-mode 14800 = iTunes Backup >= 10.0
- Added hash-mode 14900 = Skip32
- Added hash-mode 15000 = FileZilla Server >= 0.9.55

##
## Workarounds
##

- Workaround added for AMDGPU-Pro OpenCL runtime: AES encrypt and decrypt Invertkey function was calculated wrong in certain cases
- Workaround added for AMDGPU-Pro OpenCL runtime: RAR3 kernel require a volatile variable to work correctly
- Workaround added for Apple OpenCL runtime: bcrypt kernel requires a volatile variable because of a compiler optimization bug
- Workaround added for Apple OpenCL runtime: LUKS kernel requires some volatile variables because of a compiler optimization bug
- Workaround added for Apple OpenCL runtime: TrueCrypt kernel requires some volatile variables because of a compiler optimization bug
- Workaround added for NVidia OpenCL runtime: RACF kernel requires EBCDIC lookup to be done on shared memory

##
## Bugs
##

- Fixed a problem within the Kerberos 5 TGS-REP (-m 13100) hash parser
- Fixed clEnqueueNDRangeKernel(): CL_UNKNOWN_ERROR caused by an invalid work-item count during weak-hash-check
- Fixed cracking of PeopleSoft Token (-m 13500) if salt length + password length is >= 128 byte
- Fixed cracking of Plaintext (-m 99999) in case MD4 was used in a previous session
- Fixed DEScrypt cracking in BF mode in case the hashlist contains more than 16 times the same salt
- Fixed duplicate detection for WPA handshakes with the same ESSID
- Fixed nvapi datatype definition for NvS32 and NvU32
- Fixed overflow in bcrypt kernel in expand_key() function
- Fixed pointer to local variable outside scope in case -j or -k is used
- Fixed pointer to local variable outside scope in case --markov-hcstat is not used
- Fixed recursion in loopback handling when session was aborted by the user
- Fixed rule 'O' (RULE_OP_MANGLE_OMIT) in host mode in case the offset + length parameter equals the length of the input word
- Fixed rule 'i' (RULE_OP_MANGLE_INSERT) in host mode in case the offset parameter equals the length of the input word
- Fixed string not null terminated inside workaround for checking drm driver path
- Fixed string not null terminated while reading maskfiles
- Fixed truncation of password after position 32 with the combinator attack
- Fixed use of option --keyspace in combination with -m 2500 (WPA)
- Fixed WPA/WPA2 cracking in case eapol frame is >= 248 byte

##
## Technical
##

- Building: Add SHARED variable to Makefile to choose if hashcat is build as static or shared binary (using libhashcat.so/hashcat.dll)
- Building: Removed compiler option -march=native as this created problems for maintainers on various distributions
- Building: Removed the use of RPATH on linker level
- Building: Replaced linking of CRT_glob.o with the use of int _dowildcard
- Commandline: Do some checks related to custom-charset options if user specifies them
- CPU Affinity: Fixed memory leak in case invalid cpu Id was specified
- Dispatcher: Fixed several memory leaks in case an OpenCL error occurs
- Events: Improved the maximum event message handling. event_log () will now also internally make sure that the message is properly terminated
- File Locking: Improved error detection on file locks
- File Reads: Fixed memory leak in case outfile or hashfile was not accessible
- File Reads: Improved error detection on file reads, especially when getting the file stats
- Files: Do several file and folder checks on startup rather than when they are actually used to avoid related error after eventual intense operations
- Hardware Management: Bring back kernel exec timeout detection for NVidia on user request
- Hardware Monitor: Fixed several memory leaks in case hash-file writing (caused by --remove) failed
- Hardware Monitor: Fixed several memory leaks in case no hardware monitor sensor is found
- Hardware Monitor: In case NVML initialization failed, do not try to initialiaze NVAPI or XNVCTRL because they both depend on NVML
- Hash Parsing: Added additional bound checks for the SIP digest authentication (MD5) parser (-m 11400)
- Hash Parsing: Make sure that all files are correctly closed whenever a hash file parsing error occurs
- Helper: Added functions to check existence, type, read- and write-permissions and rewrite sources to use them instead of stat()
- Keyfile handling: Make sure that the memory is cleanly freed whenever a VeraCrypt/TrueCrypt keyfile fails to load
- Mask Checks: Added additional memory cleanups after parsing/verifying masks
- Mask Checks: Added integer overflow detection for a keyspace of a mask provided by user
- Mask Increment: Fixed memory leak in case mask_append() fails
- OpenCL Device: Do a check on available constant memory size and abort if it's less than 64kB
- OpenCL Device Management: Fixed several memory leaks in case initialization of an OpenCL device or platform failed
- OpenCL Header: Updated CL_* errorcode to OpenCL 1.2 standard
- OpenCL Kernel: Move kernel binary buffer from heap to stack memory
- OpenCL Kernel: Refactored read_kernel_binary to load only a single kernel for a single device
- OpenCL Kernel: Remove "static" keyword from function declarations; Causes older Intel OpenCL runtimes to fail compiling
- OpenCL Kernel: Renumbered hash-mode 7600 to 4521
- OpenCL Runtime: Added a warning about using Mesa OpenCL runtime
- OpenCL Runtime: Updated AMDGPU-Pro driver version check, do warn if version 16.60 is detected which is known to be broken
- Outfile Check: Fixed a memory leak for failed outfile reads
- Restore: Add some checks on the rd->cwd variable in restore case
- Rule Engine: Fixed several memory leaks in case loading of rules failed
- Session Management: Automatically set dedicated session names for non-cracking parameters, for example: --stdout
- Session Management: Fixed several memory leaks in case profile- or install-folder setup failed
- Sessions: Move out handling of multiple instance from restore file into separate pidfile
- Status screen: Do not try to clear prompt in --quiet mode
- Tests: Fixed the timeout status code value and increased the runtime to 400 seconds
- Threads: Restored strerror as %m is unsupported by the BSDs
- Wordlists: Disable dictstat handling for hash-mode 3000 as it virtually creates words in the wordlist which is not the case for other modes
- Wordlists: Fixed memory leak in case access a file in a wordlist folder fails
- WPA: Changed format for outfile and potfile from essid:mac1:mac2 to hash:mac_ap:mac_sta:essid
- WPA: Changed format for outfile_check from essid:mac1:mac2 to hash

* changes v3.20 -> v3.30:

##
## Features
##

- Files: Use $HEX[...] in case the password includes the separater character, increases potfile reading performance
- Files: If the user specifies a folder to scan for wordlists instead of directly a wordlist, then ignore the hidden files
- Loopback: Include passwords for removed hashes present in the potfile to next loopback iteration
- New option --progress-only: Quickly provides ideal progress step size and time to process on the user hashes and selected options, then quit
- Status screen: Reenabled automatic status screen display in case of stdin used
- Truecrypt/Veracrypt: Use CRC32 to verify headers instead of fuzzy logic, greatly reduces false positives from 18:2^48 to 3:2^64
- WPA cracking: Reuse PBKDF2 intermediate keys if duplicate essid is detected

##
## Algorithms
##

- Added hash-mode 1300 = SHA-224

##
## Bugs
##

- Fixed buffer overflow in status screen display in case of long non-utf8 string
- Fixed buffer overflow in plaintext parsing code: Leading to segfault
- Fixed custom char parsing code in maskfiles in --increment mode: Custom charset wasn't used
- Fixed display screen to show input queue when using custom charset or rules
- Fixed double fclose() using AMDGPU-Pro on sysfs compatible platform: Leading to segfault
- Fixed hash-mode 11400 = SIP digest authentication (MD5): Cracking of hashes which did not include *auth* or *auth-int* was broken
- Fixed hex output of plaintext in case --outfile-format 4, 5, 6 or 7 was used
- Fixed infinite loop when using --loopback in case all hashes have been cracked
- Fixed kernel loops in --increment mode leading to slower performance
- Fixed mask length check in hybrid attack-modes: Do not include hash-mode dependant mask length checks
- Fixed parsing of hashes in case the last line did not include a linefeed character
- Fixed potfile loading to accept blank passwords
- Fixed runtime limit: No longer required so sample startup time after refactorization

##
## Workarounds
##

- Workaround added for Intel OpenCL runtime: GPU support is broken, skip the device unless user forces to enable it

##
## Technical
##

- Building: Added hashcat32.dll and hashcat64.dll makefile targets for building hashcat windows libraries
- Building: Added production flag in Makefile to disable all the GCC compiler options needed only for development
- Building: Removed access to readlink() on FreeBSD
- Building: For CYGWIN prefer to use "opencl.dll" (installed by drivers) instead of optional "cygOpenCL-1.dll"
- Events: Added new event EVENT_WEAK_HASH_ALL_CRACKED if all hashes have been cracked during weak hash check
- Hardware management: Switched matching ADL device with OpenCL device by using PCI bus, device and function
- Hardware management: Switched matching NvAPI device with OpenCL device by using PCI bus, device and function
- Hardware management: Switched matching NVML device with OpenCL device by using PCI bus, device and function
- Hardware management: Switched matching xnvctrl device with OpenCL device by using PCI bus, device and function
- Hardware management: Removed *throttled* message from NVML as this created more confusion than it helped
- Hash Parser: Improved error detection of invalid hex characters where hex character are expected
- OpenCL Runtime: Updated AMDGPU-Pro driver version check, do warn if version 16.50 is detected which is known to be broken
- OpenCL Runtime: Updated hashcat.hctune for Iris Pro GPU on OSX
- Potfile: In v3.10 already, the default potfile suffix changed but the note about was missing. The "hashcat.pot" became "hashcat.potfile"
- Potfile: Added old potfile detection, show warning message
- Returncode: Added dedicated returncode (see docs/status_codes.txt) for shutdowns caused by --runtime and checkpoint keypress
- Sanity: Added sanity check to disallow --speed-only in combination with -i
- Sanity: Added sanity check to disallow --loopback in combination with --runtime
- Threads: Replaced all calls to ctime() with ctime_r() to ensure thread safety
- Threads: Replaced all calls to strerror() with %m printf() GNU extension to ensure thread safety

* changes v3.10 -> v3.20:

The hashcat core was completely refactored to be a MT-safe library (libhashcat).
The goal was to help developers include hashcat into distributed clients or GUI frontends.
The CLI (hashcat.bin or hashcat.exe) works as before but from a technical perspective it's a library frontend.

##
## Features
##

- New option --speed-only: Quickly provides cracking speed per device based on the user hashes and selected options, then quit
- New option --keep-guessing: Continue cracking hashes even after they have been cracked (to find collisions)
- New option --restore-file-path: Manually override the path to the restore file (useful if we want all session files in the same folder)
- New option --opencl-info: Show details about OpenCL compatible devices like an embedded clinfo tool (useful for bug reports)
- Documents: Added colors for warnings (yellow) and errors (red) instead of WARNING: and ERROR: prefix
- Documents: Added hints presented to the user about optimizing performance while hashcat is running
- Hardware management: Support --gpu-temp-retain for AMDGPU-Pro driver
- Hardware management: Support --powertune-enable for AMDGPU-Pro driver
- Password candidates: Allow words of length > 31 in wordlists for -a 0 for some slow hashes if no rules are in use
- Password candidates: Do not use $HEX[] if the password candidate is a valid UTF-8 string and print out as-is
- Pause mode: Allow quit program also if in pause mode
- Pause mode: Ignore runtime limit in pause mode
- Status view: Show core-clock, memory-clock and execution time in benchmark-mode in case --machine-readable is activated
- Status view: Show temperature, coreclock, memoryclock, fanspeed and pci-lanes for devices using AMDGPU-Pro driver
- Status view: Show the current first and last password candidate test queued for execution per device (as in JtR)
- Status view: Show the current position in the queue for both base and modifier (Example: Wordlist 2/5)
- Markov statistics: Update hashcat.hcstat which is used as reference whenever the user defines a mask
- Charsets: Added lowercase ascii hex (?h) and uppercase ascii hex (?H) as predefined charsets

##
## Algorithms
##

- Added hash-mode 14000 = DES (PT = $salt, key = $pass)
- Added hash-mode 14100 = 3DES (PT = $salt, key = $pass)
- Added hash-mode 14400 = SHA1(CX)
- Added hash-mode 99999 = Plaintext
- Extended hash-mode 3200 = bcrypt: Accept signature $2b$ (February 2014)
- Improved hash-mode 8300 = DNSSEC: Additional parsing error detection

##
## Bugs
##

- Custom charset from file parsing code did not return an error if an error occured
- Fix some clSetKernelArg() size error that caused slow modes to not work anymore in -a 1 mode
- Hash-mode 11600 = (7-Zip): Depending on input hash a clEnqueueReadBuffer(): CL_INVALID_VALUE error occured
- Hash-mode 22 = Juniper Netscreen/SSG (ScreenOS): Fix salt length for -m 22 in benchmark mode
- Hash-Mode 5500 = NetNTLMv1 + ESS: Fix loading of NetNTLMv1 + SSP hash
- Hash-mode 6000 = RipeMD160: Fix typo in array index number
- If cracking a hash-mode using unicode passwords, length check of a mask was not taking into account
- If cracking a large salted hashlist the wordlist reject code was too slow to handle it, leading to 0H/s
- Null-pointer dereference in outfile-check shutdown code when using --outfile-check-dir, leading to segfault
- On startup hashcat tried to access the folder defined in INSTALL_FOLDER, leading to segfault if that folder was not existing
- Random rules generator code used invalid parameter for memory copy function (M), leading to use of invalid rule
- Sanity check for --outfile-format was broken if used in combination with --show or --left

##
## Workarounds
##

- Workaround added for AMDGPU-Pro OpenCL runtime: Failed to compile hash-mode 10700 = PDF 1.7 Level 8
- Workaround added for AMDGPU-Pro OpenCL runtime: Failed to compile hash-mode 1800 = sha512crypt
- Workaround added for NVidia OpenCL runtime: Failed to compile hash-mode 6400 = AIX {ssha256}
- Workaround added for NVidia OpenCL runtime: Failed to compile hash-mode 6800 = Lastpass + Lastpass sniffed
- Workaround added for OSX OpenCL runtime: Failed to compile hash-mode 10420 = PDF 1.1 - 1.3 (Acrobat 2 - 4)
- Workaround added for OSX OpenCL runtime: Failed to compile hash-mode 1100 = Domain Cached Credentials (DCC), MS Cache
- Workaround added for OSX OpenCL runtime: Failed to compile hash-mode 13800 = Windows 8+ phone PIN/Password
- Workaround added for pocl OpenCL runtime: Failed to compile hash-mode 5800 = Android PIN

##
## Performance
##

- Improved performance for rule-based attacks for _very_ fast hashes like MD5 and NTLM by 30% or higher
- Improved performance for DEScrypt on AMD, from 373MH/s to 525MH/s
- Improved performance for raw DES-based algorithms (like LM) on AMD, from 1.6GH/s to 12.5GH/s
- Improved performance for raw SHA256-based algorithms using meet-in-the-middle optimization, reduces 7/64 steps
- Improved performance for SAP CODVN B (BCODE) and SAP CODVN F/G (PASSCODE) due to register handling optimization, gives 3% and 25%
- Improved performance by reducing maximum number of allowed function calls per rule from 255 to 31
- Improved performance by update the selection when to use #pragma unroll depending on OpenCL runtime vendor

- Full performance comparison sheet v3.10 vs. v3.20: https://docs.google.com/spreadsheets/d/1B1S_t1Z0KsqByH3pNkYUM-RCFMu860nlfSsYEqOoqco/edit#gid=1591672380

##
## Technical
##

- Autotune: Do not run any caching rounds in autotune in DEBUG mode if -n and -u are specified
- Bash completion: Removed some v2.01 leftovers in the bash completion configuration
- Benchmark: Do not control fan speed in benchmark mode
- Benchmark: On OSX, some hash-modes can't compile because of OSX OpenCL runtime. Skip them and move on to the next
- Building: Added Makefile target "main_shared", a small how-to-use libhashcat example
- Building: Added many additional compiler warning flags in Makefile to improve static code error detection
- Building: Added missing includes for FreeBSD
- Building: Added some types for windows only in case _BASETSD_H was not set
- Building: Changed Makefile to strip symbols in the linker instead of the compiler
- Building: Defined NOMINMAX macro to prevent definition min and max macros in stdlib header files
- Building: Enabled ASLR and DEP for Windows builds
- Building: Fixed almost all errors reported by cppcheck and scan-build
- Building: On OSX, move '-framework OpenCL' from CFLAGS to LDFLAGS
- Building: On OSX, use clang as default compiler
- Building: Support building on Msys2 environment
- Building: Use .gitmodules to simplify the OpenCL header dependency handling process
- Charsets: Added DES_full.charset
- Data Types: Replaced all integer macros with enumerator types
- Data Types: Replaced all integer variables with true bool variables in case they are used as a bool
- Data Types: Replaced all string macros with static const char types
- Data Types: Replaced all uint and uint32_t to u32
- Data Types: Replaced atoi() with atoll(). Eliminates sign conversion warnings
- Documents: Added docs/credits.txt
- Documents: Added docs/team.txt
- Documents: Changed rules.txt to match v3.20 limitations
- Error handling (file handling): Fixed a couple of filepointer leaks
- Error handling (format strings): Fixed a few printf() formats, ex: use %u instead of %d for uint32_t
- Error handling (memory allocation): Removed memory allocation checks, just print to stderr instead
- Error handling (startup): Added some missing returncode checks to get_exec_path()
- Fanspeed: Check both fanpolicy and fanspeed returncode and disable retain support if any of them fail
- Fanspeed: Minimum fanspeed for retain support increased to 33%, same as NV uses as default on windows
- Fanspeed: Reset PID controler settings to what they were initially
- Fanspeed: Set fan speed to default on quit
- File handling: Do a single write test (for files to be written later) directly on startup
- File locking: Use same locking mechanism in potfile as in outfile
- Hardware management: Fixed calling conventions for ADL, NvAPI and NVML on windows
- Hardware management: Improved checking for successfull load of the NVML API
- Hardware management: In case fanspeed can not be set, disable --gpu-temp-retain automatically
- Hardware management: In case of initialization error show it only once to the user on startup
- Hardware management: Refactored all code to return returncode (0 or -1) instead of data for more easy error handling
- Hardware management: Refactored macros to real functions
- Hardware management: Removed kernel exec timeout detection on NVIDIA, should no longer occur due to autotune
- Hardware management: Replaced NVML registry functions macros with their ascii versions (Adds NVML support for XP)
- Hashlist loading: Do not load data from hashfile if hashfile changed during runtime
- Kernel cache: Fixed checksum building on oversized device version or driver version strings
- Logging: Improved variable names in hashcat.log
- Loopback: Refactored --loopback support completely, no longer a recursive function
- Memory management: Fixed some memory leaks on shutdown
- Memory management: Got rid of all global variables
- Memory management: Got rid of local_free() and global_free(), no longer required
- Memory management: Refactored all variables with HCBUFSIZ_LARGE size from stack to heap, OSX doesn't like that
- OpenCL Headers: Select OpenCL headers tagged for OpenCL 1.2, since we use -cl-std=CL1.2
- OpenCL Kernels: Added const qualifier to variable declaration of matching global memory objects
- OpenCL Kernels: Got rid of one global kernel_threads variable
- OpenCL Kernels: Moved OpenCL requirement from v1.1 to v1.2
- OpenCL Kernels: Recognize reqd_work_group_size() values from OpenCL kernels and use them in the host if possible
- OpenCL Kernels: Refactored common function append_0x01()
- OpenCL Kernels: Refactored common function append_0x02()
- OpenCL Kernels: Refactored common function append_0x80()
- OpenCL Kernels: Refactored rule function append_block1()
- OpenCL Kernels: Refactored rule function rule_op_mangle_delete_last()
- OpenCL Kernels: Refactored rule function rule_op_mangle_dupechar_last()
- OpenCL Kernels: Refactored rule function rule_op_mangle_rotate_left()
- OpenCL Kernels: Refactored rule function rule_op_mangle_rotate_right()
- OpenCL Kernels: Support mixed kernel thread count for mixed kernels in the same source file
- OpenCL Kernels: Switch from clz() to ffz() for bitsliced algorithms
- OpenCL Kernels: Using platform vendor name is better than using device vendor name for function detection
- OpenCL Runtime: Updated AMDGPU-Pro and AMD Radeon driver version check
- OpenCL Runtime: Updated Intel OpenCL runtime version check
- OpenCL Runtime: Updated NVIDIA driver version check
- Password candidates: The maximum word length in a wordlist is 31 not 32, because 0x80 will eventually be appended
- Potfile: Base logic switched; Assuming the potfile is larger than the hashlist it's better to load hashlist instead of potfile entries
- Potfile: In case all hashes were cracking using potfile abort and inform user
- Restore: Automatically unlink restore file if all hashes have been cracked
- Restore: Do not unlink restore file if restore is disabled
- Rules: Refactored macros to real functions
- Status: Added Input.Queue.Base and Input.Queue.Mod to help the user better understand this concept
- Status: Do not wait for the progress mutex to read and store speed timer
- Status: Do not show Recovered/Time when cracking < 1000 hashes
- Status: Do not show Recovered/Time as floats but as integers to reduce over-information
- Tests: Removed rules_test/ subproject: Would require total rewrite but not used in a long time
- Threads: Replaced all calls to getpwuid() with getpwuid_r() to ensure thread safety
- Threads: Replaced all calls to gmtime() with gmtime_r() to ensure thread safety
- Threads: Replaced all calls to strtok() with strtok_r() to ensure thread safety
- Wordlists: Use larger counter variable to handle larger wordlists (that is > 2^32 words)
- X11: Detect missing coolbits and added some help text for the user how to fix it

* changes v3.00 -> v3.10:

##
## Improvements
##

- Added mask display to modes 3, 6, and 7. Allows the user to see the custom character set used during the run
- Make Linux build POSIX compatible; Also allow it to actually compile on musl-libc systems
- Add support to compile on FreeBSD
- Make use of cl_context_properties[] to clCreateContext(), even if OpenCL specification allow the use of NULL, some runtimes fail without
- The Time.Estimated attribute in status display should also show --runtime limit if user set it
- Fix some strict aliasing rule violation on older compilers
- Fix some variable initializers on older compilers
- Replace DARWIN macro with compiler predefined macro __APPLE__
- Replace LINUX macro with compiler predefined macro __linux__
- Allow the use of enc_id == 0 in hash-mode 10600 and 10700 as it takes no part in the actual computation
- Get rid of exit() calls in OpenCL wrapper library with the goal to have a better control which error can be ignored under special circumstances
- Do not error and exit if an OpenCL platform has no devices, just print a warning and continue with the next platform
- Workaround for OpenCL runtimes which do not accept -I parameter in the OpenCL kernel build options even if this is an OpenCL standard option
- Workaround for OpenCL runtimes which do accept -I parameter in the OpenCL kernel build options, but do not allow quotes
- Output cracked hashes on Windows using \r\n and not \n
- Replace RegGetValue() with RegQueryValueEx() to enable Windows XP 32 bit compatibility
- Slightly increased NVidias rule-processing performance by using generic instructions instead of byte_perm()
- Add support for @ rule (RULE_OP_MANGLE_PURGECHAR) to use on GPU
- Add support for --outfile (short -o) to be used together with --stdout
- Skip periodic status output whenever --stdout is used together with stdin mode, but no outfile was specified
- Show error message if --show is used together with --outfile-autohex-disable (this is currently not supported)
- Show error message if --skip/--limit is used together with mask files or --increment
- Workaround for NVidia OpenCL runtime bug causing -m 6223 to not crack any hashes even with the correct password candidate

##
## Bugs
##

- Fixed a bug where CRAM MD5 checked salt length instead of hash length
- Fixed a bug where hashcat is suppressing --machine-readable output in the final status update
- Fixed a bug where hashcat did not check the return of realpath() and crashes uncontrolled if the path does not exist
- Fixed a bug where hashcat crashes for accessing deallocated buffer if user spams "s" shortly before hashcat shuts down
- Fixed a bug where hashcat crashes in case of a scrypt P setting > 1
- Fixed a bug where hashcat did not correctly use the newly cracked plains whenever --loopback or the induction folder was used
- Fixed a bug where hashcat did not correctly remove hashes of type WPA/WPA2 even if present in potfile
- Fixed a bug where hashcat reported an invalid password for a zero-length password in LM
- Fixed a bug where hashcat did not take into account how long it takes to prepare a session when auto-aborting with --runtime is in use
- Fixed a bug where some kernels used COMPARE_M_SIMD instead of COMPARE_S_SIMD in singlehash mode

##
## Algorithms
##

- Added new hash-mode 13900 = OpenCart

* changes v2.01 -> v3.00:

This release markes the fusion of "hashcat" and "oclHashcat" into "hashcat".
It combines all features of all hashcat projects in one project.

##
## Features
##

- Support for Apple OpenCL runtime
- Support for NVidia OpenCL runtime (replaces CUDA)
- Support for Mesa (Gallium) OpenCL runtime
- Support for pocl OpenCL runtime
- Support for Khronos' OSS OpenCL reference implementation for building
- Support to utilize OpenCL devices-types other than GPU, ex: CPU and FPGA
- Support to utilize multiple different OpenCL platforms in parallel, ex: AMD + NV
- Support to utilize multiple different OpenCL device-types in parallel, ex: GPU + CPU
- Added option --opencl-platform to select a specific OpenCL platform
- Added option --opencl-device-types select specific OpenCL device types
- Added option --opencl-vector-width to override automatically selected vector-width size
- Added makefile native compilation target
- Added makefile install and uninstall targets
- Added autotuning engine and user-configurable tuning database
- Added current engine clock, current memory clock and pci-e lanes to the status display
- Added support for --gpu-temp-retain for NVidia GPU, both Linux and Windows
- Added execution timer of the running kernel to the status display
- Added command prompt to quit at next restore checkpoint
- Added human-readable error message for the OpenCL error codes
- Added option --potfile-path to override potfile path
- Added option --veracrypt-keyfile to set Keyfiles used, can be multiple
- Added option --veracrypt-pim to set the VeraCrypt personal iterations multiplier
- Added option --machine-readable for easier parsing of output
- Added option --powertune-enable to work with NVidia devices as well, not just AMD
- Added option --stdout to print candidates instead of trying to crack a hash

##
## Algorithms
##

- Added new hash-mode   125 = ArubaOS
- Added new hash-mode 12900 = Android FDE (Samsung DEK)
- Added new hash-mode 13000 = RAR5
- Added new hash-mode 13100 = Kerberos 5 TGS-REP etype 23
- Added new hash-mode 13200 = AxCrypt
- Added new hash-mode 13300 = AxCrypt in memory SHA1
- Added new hash-mode 13400 = Keepass 1 (AES/Twofish) and Keepass 2 (AES)
- Added new hash-mode 13500 = PeopleSoft PS_TOKEN
- Added new hash-mode 13600 = WinZip
- Added new hash-mode 137** = VeraCrypt
- Added new hash-mode 13800 = Windows 8+ phone PIN/Password

##
## Performance
##

- Full Table: https://docs.google.com/spreadsheets/d/1B1S_t1Z0KsqByH3pNkYUM-RCFMu860nlfSsYEqOoqco/edit#gid=0

##
## Improvements
##

- Reordering of files to help integration into linux distributions ~/.hashcat etc
- Use a profile directory to write temporary files (session, potfile etc.)
- Workaround dependencies on AMD APP-SDK AMD ADL, NV CUDA-SDK, NV ForceWare, NVML and NVAPI; they are no longer required
- Load external libraries dynamic at runtime instead of link them static at compile-time
- Benchmark accuracy improved; Is now on par to: singlehash -a 3 -w 3 ?b?b?b?b?b?b?b
- Benchmark no longer depends on a fixed time
- Removed option --benchmark-mode, therefore support --workload-profile in benchmark-mode
- Enabled support of --machine-readable in combination with --benchmark for automated benchmark processing
- Replaced --status-automat entirely with --machine-readable to make it more consistent among benchmark and non-benchmark mode
- Extended support from 14 to 255 functions calls per rule
- Extended password length up to 32 for 7zip
- Extended salt length up to 55 for raw hash types, eg: md5($pass.$salt)
- Extended version information
- Removed some duplicate rules in T0XlCv1, d3ad0ne and dive
- Redesigned changes.txt layout
- Redesigned --help menu layout

##
## Bugs
##

- Fixed a bug in speed display: In some situation, especially with slow hashes or lots of salts, it showed a speed of 0H/s
- Fixed a bug in restore handling: user immediately aborting after restart broke the restore file
- Fixed a bug in line counter: conditional jump or move depends on an uninitialised value
- Fixed a bug in rule-engine for NVidia devices: code for left- and right-shift were switched
- Fixed a bug in dive.rule: rules were not updated after the function 'x' was renamed to 'O'
- Fixed a bug in memory allocation "OpenCL -4 error": used unitialized value in a special situation
- Fixed a bug in memory handling: heap buffer overflow
- Fixed a bug in memory handling: out of bounds access
- Fixed a bug in implementation of DCC2: forced default iteration count for hashes to 10240
- Fixed a bug in implementation of WPA/WPA2: MAC and nonce stay one their original position as in the hccap file
- Fixed a bug in implementation of GOST R 34.11-94: zero length passwords were not cracked
- Fixed a bug in implementation of BLAKE2-512 kernels: incorrect access of the esalt buffer

##
## Technical
##

- Removed deprecated GCC version check requirement
- Removed NPROCS from Makefile, let make automatically detect the optimal number of parallel threads
- Dropped all C++ overloading functions to normal function which helps support more OpenCL platforms
- Renamed functions in common.h to emphasize their purpose
- Refactorized fast-hash kernels to enable SIMD on all OpenCL platforms
- Refactorized SIMD handling: SIMD the inner-loop not the outer-loop to save registers
- Workaround missing clEnqueueFillBuffer() support in certain OpenCL runtimes
- Added amd_bytealign() support in non-AMD OpenCL runtimes
- Added amd_bfe() support in non-AMD OpenCL runtimes
- Added several macros to allow writing optimized code for the different OpenCL platforms
- Replaced typedef for bool with stdbool.h
- Added special DEBUG environment variables to the makefile
- Hashcat now acquires an exclusive lock before writing to any file
- Changed buffers to not use same buffer for both input and output at the same time with snprintf()
- Check for allocatable device-memory depending on kernel_accel amplifier before trying to allocate
- Added additional check for max. ESSID length to prevent possible crashes
- Use a GCC equivalent for __stdcall where applicable
- Synchronize maximum output line size with input line size
- Increased maximum hash line size to 0x50000
- Run weak-hash checks only in straight-attack mode, this greatly reduces code complexity
- Restrict loopback option to straight attack-mode
- Moved rules_optimize to hashcat-utils
- Stick to older libOpenCL in binary package to avoid errors like this: version `OPENCL_2.0' not found
- Tightened hash parser for several algorithms
- Updated old RC4 code in Kerberos 5
- Limited the salt length of Juniper Netscreen/SSG (ScreenOS) hashes to 10
- Updated algorithm used to automatically select an ideal --scrypt-tmto value
- Renamed option --gpu-accel to --kernel-accel
- Renamed option --gpu-loops to --kernel-loops
- Renamed option --gpu-devices to --opencl-devices
- Added inline declaration to functions from simd.c, common.c, rp.c and types_ocl.c to increase performance
- Dropped static declaration from functions in all kernel to achieve OpenCL 1.1 compatibility
- Added -cl-std=CL1.1 to all kernel build options
- Created environment variable to inform NVidia OpenCL runtime to not create its own kernel cache
- Created environment variable to inform pocl OpenCL runtime to not create its own kernel cache
- Dropped special 64-bit rotate() handling for NV, it seems that they've added it to their OpenCL runtime
- Completely get rid of HAVE_ADL, HAVE_NVML and HAVE_NVAPI in sources
- Replaced NVAPI with NVML on windows