QuantileCutoffs

1

I found something slightly misleading in QuantileCutoffs.

I would like to compute the p99 of the income distribution, where p99 is a value such that Prob(Income>=p99) = 0.01 approximately.

I compute AllStats with simoptions.nquantiles=100. I intuitively assumed that

p99 = AllStats.Income.QuantileCutoffs(99)

but it turns out that this is the 98th percentile. So to pick the 99th percentile, I have to choose QuantileCutoffs(100). I find this slightly misleading.

Related: If I set simoptions.nquantiles=100, I would like to get p1,p2,..,p99 which are the 1st, 2nd,.., 99th percentiles of the distribution and p100 = max(Income). However, I instead get 101 values.

@robertdkirkby Just to make sure I understood things correctly, could you please confirm if this code is correct?

Params.p99 = AllStats2.Income.QuantileCutoffs(100);
Params.p97 = AllStats2.Income.QuantileCutoffs(98);
Params.p95 = AllStats2.Income.QuantileCutoffs(96);
Params.p90 = AllStats2.Income.QuantileCutoffs(91);

Instead, for the Lorenz curve, we apply the usual notation, right?

top1_wealth  = 1 - AllStats.A.LorenzCurve(99);

Thanks in advance for the clarification!

Update
If my understand is correct, the following line from your replication of B2021 is also not correct:

Params.toponepercentincomecutoff=AllStats.Income.QuantileCutoffs(99)
1 Like
2

Thanks for flagging this, @aledinola. I assumed that p50 is QuantileCutoffs(50) and so on so forth… So if I want the x percentile of a distribution I have to set QuantileCutoffs(x+1)? Is this feature documented somewhere?

I figure I will have to make some changes in my code :frowning:

3

When using AllStats there is simoptions.nquantiles=20 by default, but let’s think about what happens when the nquantiles is ‘N’.

If you have N quantiles there will be N-1 cutoffs between them.

Say you have FnsToEvaluate.Assets, and you run an AllStats command.

Then AllStats.Assets.QuantileMeans will contain N numbers, by default N=20 and these are the means for the twenty quantiles.

AllStats.Assets.QuantileCutoffs will contain N+1 numbers, by default N=20 and these are, in order: the minimum, the 19 (=20-1) cutoffs between the twenty quantiles, the maximum.

I am open to changing this behaviour if people think something else would be more appropriate.

Two possibilities occur:

  1. Just report the N-1 quantile cut-offs (so drop the min and max from current reporting). A user who asked for AllStats.Assets.QuantileCutoffs(20) would then get an error message, and on thinking about it they would probably realise that there are N-1 cutoffs and do the appropriate behaviour.
  2. Report N numbers, the min and the N-1 quantile cut-offs. This is a slightly odd mixture but would have the nice property that AllStats.Assets.QuantileCutoffs(20) is the lower cutoff of the 20th of the twenty quantiles (in default setting of N=20). This setup is an awkward mix in terms of the numbers reported, but might be nice for users as they can use without thinking.

@aledinola @jake88 what are your thoughts?

[If I change it I will make it throw warnings to tell user it has changed for about the next 12 months before then disabling the warning]

PS. Option 1 makes sense to me, as this is the quantile cutoff definition. The inclusion of min and max made sense back in the day, but now with AllStats these are anyway reported separately so doesn’t make much sense that they are still in quantile cutoffs.

1 Like
4

Hi Robert,
Thanks for your great explanation! For me the code are good as they are now, I wouldn’t make any changes. I just wanted to report this behavior and make sure I (and maybe other users) understand it correctly.

1 Like