.\" -*- mode: troff; coding: utf-8 -*- .\" Automatically generated by Pod::Man 5.01 (Pod::Simple 3.43) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. .ie n \{\ . ds C` "" . ds C' "" 'br\} .el\{\ . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" ======================================================================== .\" .IX Title "Hash::Util 3pm" .TH Hash::Util 3pm 2023-11-28 "perl v5.38.2" "Perl Programmers Reference Guide" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH NAME Hash::Util \- A selection of general\-utility hash subroutines .SH SYNOPSIS .IX Header "SYNOPSIS" .Vb 1 \& # Restricted hashes \& \& use Hash::Util qw( \& fieldhash fieldhashes \& \& all_keys \& lock_keys unlock_keys \& lock_value unlock_value \& lock_hash unlock_hash \& lock_keys_plus \& hash_locked hash_unlocked \& hashref_locked hashref_unlocked \& hidden_keys legal_keys \& \& lock_ref_keys unlock_ref_keys \& lock_ref_value unlock_ref_value \& lock_hashref unlock_hashref \& lock_ref_keys_plus \& hidden_ref_keys legal_ref_keys \& \& hash_seed hash_value hv_store \& bucket_stats bucket_info bucket_array \& lock_hash_recurse unlock_hash_recurse \& lock_hashref_recurse unlock_hashref_recurse \& \& hash_traversal_mask \& ); \& \& my %hash = (foo => 42, bar => 23); \& # Ways to restrict a hash \& lock_keys(%hash); \& lock_keys(%hash, @keyset); \& lock_keys_plus(%hash, @additional_keys); \& \& # Ways to inspect the properties of a restricted hash \& my @legal = legal_keys(%hash); \& my @hidden = hidden_keys(%hash); \& my $ref = all_keys(%hash,@keys,@hidden); \& my $is_locked = hash_locked(%hash); \& \& # Remove restrictions on the hash \& unlock_keys(%hash); \& \& # Lock individual values in a hash \& lock_value (%hash, \*(Aqfoo\*(Aq); \& unlock_value(%hash, \*(Aqfoo\*(Aq); \& \& # Ways to change the restrictions on both keys and values \& lock_hash (%hash); \& unlock_hash(%hash); \& \& my $hashes_are_randomised = hash_seed() !~ /^\e0+$/; \& \& my $int_hash_value = hash_value( \*(Aqstring\*(Aq ); \& \& my $mask= hash_traversal_mask(%hash); \& \& hash_traversal_mask(%hash,1234); .Ve .SH DESCRIPTION .IX Header "DESCRIPTION" \&\f(CW\*(C`Hash::Util\*(C'\fR and \f(CW\*(C`Hash::Util::FieldHash\*(C'\fR contain special functions for manipulating hashes that don't really warrant a keyword. .PP \&\f(CW\*(C`Hash::Util\*(C'\fR contains a set of functions that support restricted hashes. These are described in this document. \f(CW\*(C`Hash::Util::FieldHash\*(C'\fR contains an (unrelated) set of functions that support the use of hashes in \&\fIinside-out classes\fR, described in Hash::Util::FieldHash. .PP By default \f(CW\*(C`Hash::Util\*(C'\fR does not export anything. .SS "Restricted hashes" .IX Subsection "Restricted hashes" 5.8.0 introduces the ability to restrict a hash to a certain set of keys. No keys outside of this set can be added. It also introduces the ability to lock an individual key so it cannot be deleted and the ability to ensure that an individual value cannot be changed. .PP This is intended to largely replace the deprecated pseudo-hashes. .IP \fBlock_keys\fR 4 .IX Item "lock_keys" .PD 0 .IP \fBunlock_keys\fR 4 .IX Item "unlock_keys" .PD .Vb 2 \& lock_keys(%hash); \& lock_keys(%hash, @keys); .Ve .Sp Restricts the given \f(CW%hash\fR's set of keys to \f(CW@keys\fR. If \f(CW@keys\fR is not given it restricts it to its current keyset. No more keys can be added. \fBdelete()\fR and \fBexists()\fR will still work, but will not alter the set of allowed keys. \fBNote\fR: the current implementation prevents the hash from being \fBbless()\fRed while it is in a locked state. Any attempt to do so will raise an exception. Of course you can still \fBbless()\fR the hash before you call \fBlock_keys()\fR so this shouldn't be a problem. .Sp .Vb 1 \& unlock_keys(%hash); .Ve .Sp Removes the restriction on the \f(CW%hash\fR's keyset. .Sp \&\fBNote\fR that if any of the values of the hash have been locked they will not be unlocked after this sub executes. .Sp Both routines return a reference to the hash operated on. .IP \fBlock_keys_plus\fR 4 .IX Item "lock_keys_plus" .Vb 1 \& lock_keys_plus(%hash,@additional_keys) .Ve .Sp Similar to \f(CWlock_keys()\fR, with the difference being that the optional key list specifies keys that may or may not be already in the hash. Essentially this is an easier way to say .Sp .Vb 1 \& lock_keys(%hash,@additional_keys,keys %hash); .Ve .Sp Returns a reference to \f(CW%hash\fR .IP \fBlock_value\fR 4 .IX Item "lock_value" .PD 0 .IP \fBunlock_value\fR 4 .IX Item "unlock_value" .PD .Vb 2 \& lock_value (%hash, $key); \& unlock_value(%hash, $key); .Ve .Sp Locks and unlocks the value for an individual key of a hash. The value of a locked key cannot be changed. .Sp Unless \f(CW%hash\fR has already been locked the key/value could be deleted regardless of this setting. .Sp Returns a reference to the \f(CW%hash\fR. .IP \fBlock_hash\fR 4 .IX Item "lock_hash" .PD 0 .IP \fBunlock_hash\fR 4 .IX Item "unlock_hash" .PD .Vb 1 \& lock_hash(%hash); .Ve .Sp \&\fBlock_hash()\fR locks an entire hash, making all keys and values read-only. No value can be changed, no keys can be added or deleted. .Sp .Vb 1 \& unlock_hash(%hash); .Ve .Sp \&\fBunlock_hash()\fR does the opposite of \fBlock_hash()\fR. All keys and values are made writable. All values can be changed and keys can be added and deleted. .Sp Returns a reference to the \f(CW%hash\fR. .IP \fBlock_hash_recurse\fR 4 .IX Item "lock_hash_recurse" .PD 0 .IP \fBunlock_hash_recurse\fR 4 .IX Item "unlock_hash_recurse" .PD .Vb 1 \& lock_hash_recurse(%hash); .Ve .Sp \&\fBlock_hash()\fR locks an entire hash and any hashes it references recursively, making all keys and values read-only. No value can be changed, no keys can be added or deleted. .Sp This method \fBonly\fR recurses into hashes that are referenced by another hash. Thus a Hash of Hashes (HoH) will all be restricted, but a Hash of Arrays of Hashes (HoAoH) will only have the top hash restricted. .Sp .Vb 1 \& unlock_hash_recurse(%hash); .Ve .Sp \&\fBunlock_hash_recurse()\fR does the opposite of \fBlock_hash_recurse()\fR. All keys and values are made writable. All values can be changed and keys can be added and deleted. Identical recursion restrictions apply as to \fBlock_hash_recurse()\fR. .Sp Returns a reference to the \f(CW%hash\fR. .IP \fBhashref_locked\fR 4 .IX Item "hashref_locked" .PD 0 .IP \fBhash_locked\fR 4 .IX Item "hash_locked" .PD .Vb 2 \& hashref_locked(\e%hash) and print "Hash is locked!\en"; \& hash_locked(%hash) and print "Hash is locked!\en"; .Ve .Sp Returns true if the hash and its keys are locked. .IP \fBhashref_unlocked\fR 4 .IX Item "hashref_unlocked" .PD 0 .IP \fBhash_unlocked\fR 4 .IX Item "hash_unlocked" .PD .Vb 2 \& hashref_unlocked(\e%hash) and print "Hash is unlocked!\en"; \& hash_unlocked(%hash) and print "Hash is unlocked!\en"; .Ve .Sp Returns true if the hash and its keys are unlocked. .IP \fBlegal_keys\fR 4 .IX Item "legal_keys" .Vb 1 \& my @keys = legal_keys(%hash); .Ve .Sp Returns the list of the keys that are legal in a restricted hash. In the case of an unrestricted hash this is identical to calling keys(%hash). .IP \fBhidden_keys\fR 4 .IX Item "hidden_keys" .Vb 1 \& my @keys = hidden_keys(%hash); .Ve .Sp Returns the list of the keys that are legal in a restricted hash but do not have a value associated to them. Thus if 'foo' is a "hidden" key of the \f(CW%hash\fR it will return false for both \f(CW\*(C`defined\*(C'\fR and \f(CW\*(C`exists\*(C'\fR tests. .Sp In the case of an unrestricted hash this will return an empty list. .Sp \&\fBNOTE\fR this is an experimental feature that is heavily dependent on the current implementation of restricted hashes. Should the implementation change, this routine may become meaningless, in which case it will return an empty list. .IP \fBall_keys\fR 4 .IX Item "all_keys" .Vb 1 \& all_keys(%hash,@keys,@hidden); .Ve .Sp Populates the arrays \f(CW@keys\fR with the all the keys that would pass an \f(CW\*(C`exists\*(C'\fR tests, and populates \f(CW@hidden\fR with the remaining legal keys that have not been utilized. .Sp Returns a reference to the hash. .Sp In the case of an unrestricted hash this will be equivalent to .Sp .Vb 5 \& $ref = do { \& @keys = keys %hash; \& @hidden = (); \& \e%hash \& }; .Ve .Sp \&\fBNOTE\fR this is an experimental feature that is heavily dependent on the current implementation of restricted hashes. Should the implementation change this routine may become meaningless in which case it will behave identically to how it would behave on an unrestricted hash. .IP \fBhash_seed\fR 4 .IX Item "hash_seed" .Vb 1 \& my $hash_seed = hash_seed(); .Ve .Sp \&\fBhash_seed()\fR returns the seed bytes used to randomise hash ordering. .Sp \&\fBNote that the hash seed is sensitive information\fR: by knowing it one can craft a denial-of-service attack against Perl code, even remotely, see "Algorithmic Complexity Attacks" in perlsec for more information. \&\fBDo not disclose the hash seed\fR to people who don't need to know it. See also "PERL_HASH_SEED_DEBUG" in perlrun. .Sp Prior to Perl 5.17.6 this function returned a UV, it now returns a string, which may be of nearly any size as determined by the hash function your Perl has been built with. Possible sizes may be but are not limited to 4 bytes (for most hash algorithms) and 16 bytes (for siphash). .IP \fBhash_value\fR 4 .IX Item "hash_value" .Vb 2 \& my $hash_value = hash_value($string); \& my $hash_value = hash_value($string, $seed); .Ve .Sp \&\f(CWhash_value($string)\fR returns the current perl's internal hash value for a given string. \&\f(CW\*(C`hash_value($string, $seed)\*(C'\fR returns the hash value as if computed with a different seed. If the custom seed is too short, the function errors out. The minimum length of the seed is implementation-dependent. .Sp Returns a 32\-bit integer representing the hash value of the string passed in. The 1\-parameter value is only reliable for the lifetime of the process. It may be different depending on invocation, environment variables, perl version, architectures, and build options. .Sp \&\fBNote that the hash value of a given string is sensitive information\fR: by knowing it one can deduce the hash seed which in turn can allow one to craft a denial-of-service attack against Perl code, even remotely, see "Algorithmic Complexity Attacks" in perlsec for more information. \&\fBDo not disclose the hash value of a string\fR to people who don't need to know it. See also "PERL_HASH_SEED_DEBUG" in perlrun. .IP \fBbucket_info\fR 4 .IX Item "bucket_info" Return a set of basic information about a hash. .Sp .Vb 1 \& my ($keys, $buckets, $used, @length_counts)= bucket_info($hash); .Ve .Sp Fields are as follows: .Sp .Vb 5 \& 0: Number of keys in the hash \& 1: Number of buckets in the hash \& 2: Number of used buckets in the hash \& rest : list of counts, Kth element is the number of buckets \& with K keys in it. .Ve .Sp See also \fBbucket_stats()\fR and \fBbucket_array()\fR. .IP \fBbucket_stats\fR 4 .IX Item "bucket_stats" Returns a list of statistics about a hash. .Sp .Vb 3 \& my ($keys, $buckets, $used, $quality, $utilization_ratio, \& $collision_pct, $mean, $stddev, @length_counts) \& = bucket_stats($hashref); .Ve .Sp Fields are as follows: .Sp .Vb 10 \& 0: Number of keys in the hash \& 1: Number of buckets in the hash \& 2: Number of used buckets in the hash \& 3: Hash Quality Score \& 4: Percent of buckets used \& 5: Percent of keys which are in collision \& 6: Mean bucket length of occupied buckets \& 7: Standard Deviation of bucket lengths of occupied buckets \& rest : list of counts, Kth element is the number of buckets \& with K keys in it. .Ve .Sp See also \fBbucket_info()\fR and \fBbucket_array()\fR. .Sp Note that Hash Quality Score would be 1 for an ideal hash, numbers close to and below 1 indicate good hashing, and number significantly above indicate a poor score. In practice it should be around 0.95 to 1.05. It is defined as: .Sp .Vb 4 \& $score= sum( $count[$length] * ($length * ($length + 1) / 2) ) \& / \& ( ( $keys / 2 * $buckets ) * \& ( $keys + ( 2 * $buckets ) \- 1 ) ) .Ve .Sp The formula is from the Red Dragon book (reformulated to use the data available) and is documented at .IP \fBbucket_array\fR 4 .IX Item "bucket_array" .Vb 1 \& my $array= bucket_array(\e%hash); .Ve .Sp Returns a packed representation of the bucket array associated with a hash. Each element of the array is either an integer K, in which case it represents K empty buckets, or a reference to another array which contains the keys that are in that bucket. .Sp \&\fBNote that the information returned by bucket_array is sensitive information\fR: by knowing it one can directly attack perl's hash function which in turn may allow one to craft a denial-of-service attack against Perl code, even remotely, see "Algorithmic Complexity Attacks" in perlsec for more information. \&\fBDo not disclose the output of this function\fR to people who don't need to know it. See also "PERL_HASH_SEED_DEBUG" in perlrun. This function is provided strictly for debugging and diagnostics purposes only, it is hard to imagine a reason why it would be used in production code. .IP \fBbucket_stats_formatted\fR 4 .IX Item "bucket_stats_formatted" .Vb 1 \& print bucket_stats_formatted($hashref); .Ve .Sp Return a formatted report of the information returned by \fBbucket_stats()\fR. An example report looks like this: .Sp .Vb 12 \& Keys: 50 Buckets: 33/64 Quality\-Score: 1.01 (Good) \& Utilized Buckets: 51.56% Optimal: 78.12% Keys In Collision: 34.00% \& Chain Length \- mean: 1.52 stddev: 0.66 \& Buckets 64 [0000000000000000000000000000000111111111111111111122222222222333] \& Len 0 Pct: 48.44 [###############################] \& Len 1 Pct: 29.69 [###################] \& Len 2 Pct: 17.19 [###########] \& Len 3 Pct: 4.69 [###] \& Keys 50 [11111111111111111111111111111111122222222222222333] \& Pos 1 Pct: 66.00 [#################################] \& Pos 2 Pct: 28.00 [##############] \& Pos 3 Pct: 6.00 [###] .Ve .Sp The first set of stats gives some summary statistical information, including the quality score translated into "Good", "Poor" and "Bad", (score<=1.05, score<=1.2, score>1.2). See the documentation in \&\fBbucket_stats()\fR for more details. .Sp The two sets of barcharts give stats and a visual indication of performance of the hash. .Sp The first gives data on bucket chain lengths and provides insight on how much work a fetch *miss* will take. In this case we have to inspect every item in a bucket before we can be sure the item is not in the list. The performance for an insert is equivalent to this case, as is a delete where the item is not in the hash. .Sp The second gives data on how many keys are at each depth in the chain, and gives an idea of how much work a fetch *hit* will take. The performance for an update or delete of an item in the hash is equivalent to this case. .Sp Note that these statistics are summary only. Actual performance will depend on real hit/miss ratios accessing the hash. If you are concerned by hit ratios you are recommended to "oversize" your hash by using something like: .Sp .Vb 1 \& keys(%hash)= keys(%hash) << $k; .Ve .Sp With \f(CW$k\fR chosen carefully, and likely to be a small number like 1 or 2. In theory the larger the bucket array the less chance of collision. .IP \fBhv_store\fR 4 .IX Item "hv_store" .Vb 4 \& my $sv = 0; \& hv_store(%hash,$key,$sv) or die "Failed to alias!"; \& $hash{$key} = 1; \& print $sv; # prints 1 .Ve .Sp Stores an alias to a variable in a hash instead of copying the value. .IP \fBhash_traversal_mask\fR 4 .IX Item "hash_traversal_mask" As of Perl 5.18 every hash has its own hash traversal order, and this order changes every time a new element is inserted into the hash. This functionality is provided by maintaining an unsigned integer mask (U32) which is xor'ed with the actual bucket id during a traversal of the hash buckets using \fBkeys()\fR, \&\fBvalues()\fR or \fBeach()\fR. .Sp You can use this subroutine to get and set the traversal mask for a specific hash. Setting the mask ensures that a given hash will produce the same key order. \fBNote\fR that this does \fBnot\fR guarantee that \fBtwo\fR hashes will produce the same key order for the same hash seed and traversal mask, items that collide into one bucket may have different orders regardless of this setting. .IP \fBbucket_ratio\fR 4 .IX Item "bucket_ratio" This function behaves the same way that scalar(%hash) behaved prior to Perl 5.25. Specifically if the hash is tied, then it calls the SCALAR tied hash method, if untied then if the hash is empty it return 0, otherwise it returns a string containing the number of used buckets in the hash, followed by a slash, followed by the total number of buckets in the hash. .Sp .Vb 2 \& my %hash=("foo"=>1); \& print Hash::Util::bucket_ratio(%hash); # prints "1/8" .Ve .IP \fBused_buckets\fR 4 .IX Item "used_buckets" This function returns the count of used buckets in the hash. It is expensive to calculate and the value is NOT cached, so avoid use of this function in production code. .IP \fBnum_buckets\fR 4 .IX Item "num_buckets" This function returns the total number of buckets the hash holds, or would hold if the array were created. (When a hash is freshly created the array may not be allocated even though this value will be non-zero.) .SS "Operating on references to hashes." .IX Subsection "Operating on references to hashes." Most subroutines documented in this module have equivalent versions that operate on references to hashes instead of native hashes. The following is a list of these subs. They are identical except in name and in that instead of taking a \f(CW%hash\fR they take a \f(CW$hashref\fR, and additionally are not prototyped. .IP lock_ref_keys 4 .IX Item "lock_ref_keys" .PD 0 .IP unlock_ref_keys 4 .IX Item "unlock_ref_keys" .IP lock_ref_keys_plus 4 .IX Item "lock_ref_keys_plus" .IP lock_ref_value 4 .IX Item "lock_ref_value" .IP unlock_ref_value 4 .IX Item "unlock_ref_value" .IP lock_hashref 4 .IX Item "lock_hashref" .IP unlock_hashref 4 .IX Item "unlock_hashref" .IP lock_hashref_recurse 4 .IX Item "lock_hashref_recurse" .IP unlock_hashref_recurse 4 .IX Item "unlock_hashref_recurse" .IP hash_ref_unlocked 4 .IX Item "hash_ref_unlocked" .IP legal_ref_keys 4 .IX Item "legal_ref_keys" .IP hidden_ref_keys 4 .IX Item "hidden_ref_keys" .PD .SH CAVEATS .IX Header "CAVEATS" Note that the trapping of the restricted operations is not atomic: for example .PP .Vb 1 \& eval { %hash = (illegal_key => 1) } .Ve .PP leaves the \f(CW%hash\fR empty rather than with its original contents. .SH BUGS .IX Header "BUGS" The interface exposed by this module is very close to the current implementation of restricted hashes. Over time it is expected that this behavior will be extended and the interface abstracted further. .SH AUTHOR .IX Header "AUTHOR" Michael G Schwern on top of code by Nick Ing-Simmons and Jeffrey Friedl. .PP \&\fBhv_store()\fR is from Array::RefElem, Copyright 2000 Gisle Aas. .PP Additional code by Yves Orton. .PP Description of \f(CW\*(C`hash_value($string, $seed)\*(C'\fR by Christopher Yeleighton .SH "SEE ALSO" .IX Header "SEE ALSO" Scalar::Util, List::Util and "Algorithmic Complexity Attacks" in perlsec. .PP Hash::Util::FieldHash.