Читать книгу Applied Modeling Techniques and Data Analysis 2 - Группа авторов - Страница 18

Our first model predicts, on the basis of the available features, 415 taxpayers to be interesting (i.e. 15.5% of the entire test set), with a precision rate of about 80%, as shown in Figure 1.6.

Figure 1.6. First model statistics and confusion matrix

In terms of tax claim amounts, the model appears to perform quite well, since the selected taxpayers’ average due additional taxes amounts to € 49,094, whereas the average on the entire test set is equal to € 22,339.

So far, we have shown that our model, on average, is able to distinguish serious tax evasion phenomena from the less significant ones. But what about the tax collection issue? To deal with this matter, we should investigate what kind of taxpayers we have just selected. For this purpose, Table 1.3 shows that the majority of the taxpayers, the model would select, would also be subject to coercive procedures (as we can see, the sum of the values of each column is 100%).

Table 1.3. Predicted values versus actual coercive procedures

	Pred Interesting Not Interesting
Act
Procedure	70.12%	32.24%
No procedure	29.88%	67.76%

Thus, many of the selected taxpayers have a debt payment issue. This jeopardizes the overall selection process efficiency and effectiveness. As pointed out by the Italian Court of Auditors, coercive procedures, on average, are able to collect only about 5% of the overall claimed credits.

To evaluate the problem extent, we can replace the actual tax claim value corresponding to the problematic taxpayers with the estimated collectable tax, which is equal to the tax claim multiplied by a discount factor of 95%, and compare the two scenarios, as in Figures 1.7 and 1.8, where we depict both the total tax claim and the average tax claim arising from the taxpayers’ notices in the entire test set.

Figure 1.7. Total tax claim and discounted tax claim. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

Taxpayers are ordered, from left to right, according to their probability of being interesting, as calculated by our model. Figure 1.7, for instance, depicts the cumulative tax claim charged up to a certain taxpayer: the red line values refer to the additional taxes requested with the tax notices, while the black line is drawn by considering the discounted values. The dashed vertical line indicates the levels corresponding to the last selected taxpayer according to the model (in our case, the 415th). Recall that when associating a class label with a record, the model also provides a probability, which highlights how confident the model is about its own prediction. Therefore, to a certain extent, it sets a ranking among taxpayers, which we can exploit to draw Figures 1.7 and 1.8. As we can easily observe, the overall tax claim charged to the selected taxpayers plummets from € 20 million to € 5 million, and the average tax claim, depicted in Figure 1.8, from € 49,000 to € 12,000. Thus, the selection process, which relied on our data mining model and at first sight seemed to be very efficient, shows some important flaws that we need to face. In fact, tax collectability is not adequately guaranteed.

Figure 1.8. Average total tax claim and discounted tax claim. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

A second model may then help us by predicting which taxpayers would not be subject to coercive procedures, by focusing on a set of features concerning their assets.

Again, with a precision rate of about 80%, as shown in Figure 1.9, the model appears to be successful.

Figure 1.9. Second model statistics and confusion matrix

Table 1.4. Predicted coercive procedures versus actual interesting taxpayers

	Pred Procedure No Procedure
Act
Interesting	46.94%	32.73%
Not interesting	53.06%	67.27%

This second model could be useful on our end, even though it may have some caveats. First, most of the taxpayers that the model classifies as people that will not face a coercive procedure are also not interesting, as shown in Table 1.4. Again, the sum of the values of each column is 100%.

In fact, this second model’s performance in terms of tax claim appears to have worsened with respect to the first, since the no procedure taxpayers’ average due additional tax, calculated on the first 415 taxpayers (according to the ranking set by this model, which is, obviously, dramatically different from the one set by the first model we have seen), is equal to € 20,388. However, the average collectable tax claim is equal to € 13,493, which is a little bit better than the one we have seen before.

We point out that throughout this chapter, we have compared sets of selected taxpayers with the same cardinality, for two kinds of considerations: first, tax authorities, reasonably, have a fixed budget of audits to perform, so comparisons between models should be done subject to a given number of audits; second, for comparability reasons, since smaller sets tend to perform more (see Figure 1.8, where the average tax claim decreases while the number of selected taxpayers increases).

Therefore, in this second model we have developed, the high rate of not interesting taxpayers, on one hand, causes a drop in the average tax claim (from 49,000 to 20,000), but, on the other, it contributes to the slight enhancement of the discounted average tax claim (from € 12,000 to € 13,000), since only a few of the not interesting taxpayers pass through a coercive procedure. Figure 1.10 compares, for each number of selected taxpayers, the different coercive procedures rates arising from the two models.

What we can do, then, is use the two models “together”. For instance, we could exploit the first model in order to sort the taxpayers eligible to be selected and the second one to discard the ones likely to be subject to coercive procedures.

In such a way, if we imagine that we select our 415 taxpayers again, on one hand, we would select both interesting and not interesting taxpayers (only if the second model had predicted that no interesting taxpayers would go through a coercive procedure, we would have selected only interesting taxpayers), but, on the other, we would also select the taxpayers who are more likely to pay their tax debts.

Figure 1.10. Coercive procedures’ rates. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

This is just an example and it is not the only way we can combine the two models. Indeed, there is space for policymakers to exploit the two models in different ways, depending on the kind of tradeoff choices they may want to reach, concerning the two goals of the audit process: its profitability and its tax collectability. For instance, a selection process could only be targeted towards interesting taxpayers and taxpayers without payment issues.

Anyway, does the tradeoff we have sketched above work?

Figures 1.11-1.13 can shed some light on our ensemble model’s performance. As usual, the dashed vertical line shows the values corresponding to the number of taxpayers we wish to select.

In our case, thus, with the ensemble model, we would claim, on average, € 26,219 from the selected taxpayers and we would hopefully collect, on average, € 17,542 from each of them, of whom only 25% are predicted to incur in coercive procedures.

Figure 1.11. Total tax claim. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

Figure 1.12. Average tax claim. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

Figure 1.13. Coercive procedures’ rate. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

In a hypothetical selection process, the winning strategy would then be to use the ensemble model, since it maximizes the collectable tax claim.

What we might be interested in, is to know whether the ensemble model is always the best option. This may depend on the coercive procedures’ rate that characterizes the two sets of auditable taxpayers selected by the two models. Unfortunately, once we build the models, before applying them to the test set, we do not exactly know what kind of taxpayers will be selected. Therefore, we do not even know these rates; however, we can consider them as unknown parameters, say θ’ and θ”. From this point of view, the rates we have observed within the two selected sets can be considered as two values of such parameters, say (70%) and (25%) (see Table 1.5).

To satisfy our interest, we should depict the two models’ behavior as a function of the unknown parameters, θ’ and θ”, respectively; that is, we should calculate the expected tax revenues amounts for any value of θ’ and θ”. Unfortunately, this cannot be done. To understand why, suppose that for both models, only one of the selected taxpayers turns out to be subject to coercive procedures. If this taxpayer’s debt is high, the amount of money that is difficult to collect would be high, but if his debt is low, then the uncollected tax would also be low.

What can be done, instead, is to calculate, for any given value of θ’ and θ”, the maximum and minimum collectable taxes arising from each model. Indeed, the maximum collectable taxes scenario is the one where coercive procedures are first applied to the less unfaithful taxpayers, while the minimum collectable taxes scenario refers to a situation in which coercive procedures are first applied to the most unfaithful taxpayers, as shown in Figure 1.14.

Figure 1.14.Models’ maximum and minimum collected tax. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

The first model’s maximum and minimum values are represented by the red and orange lines, while the ensemble model’s are the blue and purple ones. Any point within the red and orange lines represents a possible outcome for the first model and any point within the blue and purple lines represents one possible outcome for the ensemble model. For instance, points A and B represent the outcomes of our models (the first and the ensemble, respectively), given our training and test sets.

Having to deal with two areas means that the models’ behavior is determined not only by θ’ and θ”, but also by the kind of taxpayers that go through a coercive procedure. If we could shrink the areas between the red and orange lines and the blue and purple ones, we could be put in a better shape.

How could we do this? Well, if we turn back to points A and B in Figure 1.14, and we draw two dashed vertical lines from them, we can see that the first is nearer to the minimum line of its model (since line is shorter than line ), while the other is nearer to the maximum one (since line is shorter than line ).

If we assume that, for each value of θ’ and θ” and for each corresponding points A (and, also, lines and ) and B (and, also, lines and ), ratios are always the same and also ratios , we could draw a single line for each model, which would only be a function of θ’ and θ’’, respectively, as shown in Figure 1.15.

Figure 1.15. Models’ approximation. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

Therefore, we have to study the two monotonically decreasing functions, say γ_first(θ’”) and γ_ens(θ”) to find out for which joint values of θ’ and θ”, one model is better than the other.

Based on our data, these functions intersect at two points, where θ’ and θ” are, respectively, equal to α and β. Moreover:

– γ_first(0) > γ_ens (0), i.e. if all taxpayers were to pay their debts, the first model would be better than the ensemble one.

– γ_first(1) > γ_ens (1) since if all taxpayers were to undergo a coercive procedure, these functions’ values would be 0.05 times γ_first(0) and γ_ens (0), respectively (recall that in the case of coercive procedures, the collectable tax is assumed to be equal to the tax claim multiplied by a discount factor of 95%).

– There is a ψ such that γf_irst(θ‘) ≥ γ_ens(θ”), for θ’ ≤ ψ and for any θ”.

– There is a ø such that Yfirst(θ’)≥ γ_ens(θ’’), for θ’ ≥ ø and for any θ“.

Figure 1.16 depicts, in a θ’ x θ” space, the regions where the two models represent the best choice (the dark gray region is where the first model is the best option, while in the light gray one, the ensemble model is better).

Figure 1.16. Values of θ’ and θ” determining the best model. For a color version of this figure, see www.iste.co.uk/dimotikalis/analysis2.zip

In the three white regions, the exact combinations of θ’ and θ” that guarantee whether a model is better than the other, depend on the relative slopes of γf_irst(θ’) and γens(θ”).

As a general rule, if we expect small values of θ’ or high values of θ‘, and also high values of θ’’ in our samples of auditable taxpayers, then the first model is likely to guarantee a higher revenue; otherwise, the ensemble model is the one that we should use. From Figure 1.16, we note that our experience on the 8,000 taxpayers dataset took us to point Γ, which lies in a region where the ensemble model is the best option.

Table 1.5. The most significant results of the models

	First model	Second model	Ensemble model	Test set
Number of selected taxpayers	415	415	415	2,676
Interesting taxpayers rate	82.20%	32.77%	42.89%	43.12%
Coercive procedures rate	70.12%	17.35%	24.58%	38.12%
Average tax claim (€)	49,094	20,388	26,219	22,339
Average collectable tax (€)	12,187	13,493	17,542	10,194