Repository navigation
Q: Idiomatic way to specify the length of an arbitrary vector #297
Description
Is there documentation on the idiomatic way to specify the length of an arbitrary vector?
My specific example is this:
I have a struct with fields containing various data types (like max 64-bytes, max 255-bytes, exactly 32-bytes, etc.) that I need to implement Arbitrary for. As an example, I will use one field in the Foo struct that represents a vector of 255 bytes.
struct Foo {
bar: B0255
}
Where
struct B0255<'b>(Inner<'b>);
enum Inner<'a> {
Ref(&'a [u8]),
Owned(Vec<u8>),
}
To test this struct, I have created a new struct and implemented Arbitrary, which calls a function from_random which is implemented for Foo. from_random is needed because Arbitrary needs to be implemented in another crate.
struct RandomFoo(Foo);
impl Arbitrary from RandomFoo {
fn arbitrary(g: &mut Gen) -> Self {
RandomFoo(Foo::from_random(g))
}
}
Currently, I am passing in the generator to the from_random function and generating the 255 byte vectors as:
impl Foo {
fn from_random(g: &mut Gen) -> Self {
let mut bar = Vec<u8>::arbitrary(g);
bar.truncate(255);
let bar: B0255 = bar.try_into().unwrap();
Foo { bar }
}
}
Here I am truncating the vector to avoid a panic caused by a vector that is too big. Was thinking that the best approach may be having the try_into return the right error when called with a value that is too big, but I am not sure if this is possible as Arbitrary needs to return Self?
Additionally, when from_random is implemented for fixed size primitives (eg U256) should resize(255, 0) be used instead of truncate?
Alternatively, I can implement from_random as:
impl Foo {
fn from_random(g: &mut Gen) -> Self {
let mut bar: B0255 = Gen::new(255);
bar: B0255 = Vec::<u8>::arbitrary(&mut bar).try_into().unwrap();
Foo { bar }
}
}
Which, if either, from_random implementation is the idiomatic way to generate a vector of a specific size?
Activity
neithernut commentedon Aug 13, 2021
In the past, I used something like the following in such situations:
Obviously not the most pretty thing to write. And you must take care not to forget wrapping
Arbitrary::arbitrary(g), otherwise you'll end up with fewer elements than you expected. On the plus side, you definitely don't generate any values you'll truncate away anyway.If/when #292 gets merged, you can also use the following pattern:
One day, #282 may also provide an option if your target size is known at compile time:
neithernut commentedon Aug 13, 2021
Stupid me just realized that you can just do:
Fi3 commentedon Aug 13, 2021
@neithernut ty!
So you are suggesting to create a new generator in the
from_randomfunction and do not use the one available inArbitrary::arbitrary?Can it give any problem when implementing
shrink?Asking cause right now we are using the generic
shrinkbut we plan in the future to implement a correctshrinkso we would come out with a solution that play well with it.neithernut commentedon Aug 15, 2021
It depends. With recursive structures, you usually want to either create a new generator with a reduced size in order to ensure the recursion is bounded. In such cases, you also want to specialcase the generator having a size of
0. But if you want to generate aVec<u8>or aVecof some other trivial type there's no need to create a new generator. Creating a new generator with the current interfaces also comes with the caveat that you loose reproducibility.Whether or not you create a new generator doesn't affect shrinking at all. Shrinking in itself is both independent from the generator and deterministic (unless you do very weird things in your shrinker).
Fi3 commentedon Aug 19, 2021
Ok. Is not very clear why should I shrink if I lose reproducibility? Is not goal of shrinking to have a minimal failing case? If I do not have reproducibility maybe the shrinked case is not failing anymore. Just asking for future reference, right now I will implement from random following your above example, and use the default shrink function.
Ty again
neithernut commentedon Aug 19, 2021
You loose reproducibility in the generation when creating a new
Geninstance for sub-structures. Which is, btw., only relevant if you control or extract the seed used to create the initialGenin the first place. But still, any test should be reproducible for a given value. It's only the generation itself that will lack reproducibility.Shrinking should always be deterministic and only operate on the value itself. For example, you should never use
Arbitrary::arbitrary,Genor any other source of randomness in your shrinker.Sorry if I wasn't clear enough on that front.
quickcheckwill apply the test on every value yielded by the shrinker, select a value for which the test (still) fails and repeat (by shrinking that value) until the failure can't be reproduced any more. The whole process works under the assumption that a test is reproducible for given inputs as explained above. So, ifquickcheckreports a shrunk value, that value will reproduce the failure (if said assumptions hold).Note that the default shrink function supplied by the
Arbitrarytrait will not yield any shrunk values, i.e.quichcheckwill always report the initial reproducer for which a test failed. This may be a valid choice if your value is simple and/or can't be shrunk realistically.The initial post by @rrybarczyk talks about a
Vec<u8>with a fixed lengh. As theVec's length should (probably) be constant under shrinking, the only reasonable strategy appears to shrink each individual value in thatVec. The shrinker would then returnVecs for which a single position differs from the original (we'd let the aforementioned repetition take care of the rest). Depending on what exactly you test for, such a shrinker may not be worth the effort.rrybarczyk commentedon Aug 20, 2021
Thank you so much @neithernut!