Hi Jcatanz, answering your questions

1. What I call test data is really validation data for which you have target. Should have stuck to using validation data to avoid this confusion.
2. Good catch! That happens because binning currently is not dynamic enough. Say, you were creating 10 bins. It first calculates 10, 20, 30, etc. percentile values for that feature. Now let’s say first 18% of customers (sorted by feature value) have same special value. This 18% becomes first bin and only 2% customers are left for second bin which leads to one small sample sized bin.