Читать книгу Machine Learning for Tomographic Imaging - Professor Ge Wang - Страница 55

The cross entropy (De Boer et al 2002) is often used in the case of binary classification, in which labels are assumed to take values 0 or 1. As a loss function, the cross entropy is computed by the following equation when y and yˆ are probability functions:

L=−1n∑i=1ny(i)logyˆ(i)+1−y(i)log1−yˆ(i).(3.22)

The cross entropy measures the divergence between the probability distributions of predicted and real results. A large cross entropy value means that the two distributions are clearly distinct, while small cross entropy values suggest that the two distributions are close to each other. Compared to the quadratic cost function, the cross entropy often enjoys fast convergence and is more likely associated with the global optimization.

The idea of cross entropy can be extended to the cases of multi-classification tasks, leading to the concept of the multi-class cross entropy (see appendix A for more details).